We theoretically study the optical generation of dc spin current (i.e., a spin-current solar cell) in ordered antiferromagnetic and ferrimagnetic insulators, motivated by a recent study on the laser-driven spinon spin current in noncentrosymmetric quantum spin chains [H. Ishizuka and M. Sato, Phys. Rev. Lett. 122, 197702 (2019)]. Using a non-linear response theory for magnons, we analyze the dc spin current generated by a linearly-polarized electromagnetic wave (typically, terahertz or gigahertz waves). Considering noncentrosymmetric two-sublattice magnets as an example, we find a finite dc spin current conductivity at $T=0$, where no thermally-excited magnons exist; this is in contrast to the case of the spinon spin current, in which the optical transition of the Fermi degenerate spinons plays an essential role. We find that the dc spin-current conductivity is insensitive to the Gilbert damping, i.e., it may be viewed as a shift current carried by bosonic particles (magnons). Our estimate shows that an electric-field intensity of $Esim10^4-10^6$ V/cm is sufficient for an observable spin current. Our theory indicates that the linearly-polarized electromagnetic wave generally produces a dc spin current in noncentrosymmetric magnetic insulators.