LiOsO$_3$ is the first experimentally confirmed polar metal with ferroelectric-like distortion. One puzzling experimental fact is its paramagnetic state down to very low temperature with negligible magnetic moment, which is anomalous considering its $5d^3$ electron configuration since other osmium oxides (e.g. NaOsO$_3$) with $5d^3$ Os ions are magnetic. Here the magnetic and electronic properties of LiOsO$_3$ are re-investigated carefully using the first-principles density functional theory. Our calculations reveal that the magnetic state of LiOsO$_3$ can be completely suppressed by the spin-orbit coupling. The subtle balance between significant spin-orbit coupling and weak Hubbard $U$ of $5d$ electrons can explain both the nonmagnetic LiOsO$_3$ and magnetic NaOsO$_3$. Our work provides a reasonable understanding of the long-standing puzzle of magnetism in some osmium oxides.