The detailed magnetic field structure of the starless dense core CB81 (L1774, Pipe 42) in the Pipe Nebula was determined based on near-infrared polarimetric observations of background stars to measure dichroically polarized light produced by magnetically aligned dust grains in the core. The magnetic fields pervading CB81 were mapped using 147 stars and axisymmetrically distorted hourglass-like fields were identified. On the basis of simple 2D and 3D magnetic field modeling, the magnetic inclination angles in the plane-of-sky and line-of-sight directions were determined to be $4^{circ} pm 8^{circ}$ and $20^{circ} pm 20^{circ}$, respectively. The total magnetic field strength of CB81 was found to be $7.2 pm 2.3$ $mu{rm G}$. Taking into account the effects of thermal/turbulent pressure and magnetic fields, the critical mass of CB81 was calculated to be $M_{rm cr}=4.03 pm 0.40$ M$_{odot}$, which is close to the observed core mass of $M_{rm core}=3.37 pm 0.51$ M$_{odot}$. We thus conclude that CB81 is in a condition close to the critical state. In addition, a spatial offset of $92$ was found between the center of magnetic field geometry and the dust extinction distribution; this offset structure could not have been produced by self-gravity. The data also indicate a linear relationship between polarization and extinction up to $A_V sim 30$ mag going toward the core center. This result confirms that near-infrared polarization can accurately trace the overall magnetic field structure of the core.