Recent optical second-harmonic generation experiments observed unexpected broken spatial symmetries in the undoped spin-orbit Mott insulator Sr$_2$IrO$_4$, leading to intensive debates on the nature of its ground state. We propose that it is a canted antiferromagnetism with a hidden order of circulating staggered pseudospin current. Symmetry analysis shows that a proper $c$-axis stacking of the canted antiferromagnetism and the pseudospin current lead to a magnetoelectric coexistence state that breaks the two-fold rotation, inversion, and time-reversal symmetries, consistent with experimental observations. We construct a three-dimensional Hubbard model with spin-orbit coupling for the five localized 5$d$ Wannier orbitals centered at Ir sites, and demonstrate the microscopic realization of the desired coexistence state in a wide range of band parameters via a combination of self-consistent Hartree-Fock and variational calculations.