The mass flow rate of Poiseuille flow of rarefied gas through long ducts of two-dimensional cross-sections with arbitrary shape are critical in the pore-network modeling of gas transport in porous media. In this paper, for the first time, the high-order hybridizable discontinuous Galerkin (HDG) method is used to find the steady-state solution of the linearized Bhatnagar-Gross-Krook equation on two-dimensional triangular meshes. The velocity distribution function and its traces are approximated in the piecewise polynomial space (of degree up to 4) on the triangular meshes and the mesh skeletons, respectively. By employing a numerical flux that is derived from the first-order upwind scheme and imposing its continuity on the mesh skeletons, global systems for unknown traces are obtained with a few coupled degrees of freedom. To achieve fast convergence to the steady-state solution, a diffusion-type equation for flow velocity that is asymptotic-preserving into the fluid dynamic limit is solved by the HDG simultaneously, on the same meshes. The proposed HDG-synthetic iterative scheme is proved to be accurate and efficient. Specifically, for flows in the near-continuum regime, numerical simulations have shown that, to achieve the same level of accuracy, our scheme could be faster than the conventional iterative scheme by two orders of magnitude, while it is faster than the synthetic iterative scheme based on the finite difference discretization in the spatial space by one order of magnitude. The HDG-synthetic iterative scheme is ready to be extended to simulate rarefied gas mixtures and the Boltzmann collision operator.