In this paper we report low-temperature magnetic properties of the rare-earth perovskite material YbAlO$_3$. Results of elastic and inelastic neutron scattering experiment, magnetization measurements along with the crystalline electrical field (CEF) calculations suggest that the ground state of Yb moments is a strongly anisotropic Kramers doublet, and the moments are confined in the $ab$-plane, pointing at an angle of $varphi = pm 23.5^{circ}$ to the $a$-axis. With temperature decreasing below $T_{rm N}=0.88$ K, Yb moments order into the coplanar, but non-collinear antiferromagnetic (AFM) structure $AxGy$, where the moments are pointed along their easy-axes. In addition, we highlight the importance of the dipole-dipole interaction, which selects the type of magnetic ordering and may be crucial for understanding magnetic properties of other rare-earth orthorhombic perovskites. Further analysis of the broad diffuse neutron scattering shows that one-dimensional interaction along the $c$-axis is dominant, and suggests YbAlO$_3$ as a new member of one dimensional quantum magnets.