In the presence of both space and time reversal symmetries, an s-wave A1g superconducting state is usually topologically trivial. Here we demonstrate that an exception can take place in a type of nonsymmorphic lattice structures. We specify the demonstration in a system with a centrosymmetric space group P4/nmm, the symmetry group that governs iron-based superconductors, by showing the existence of a second-order topological state protected by a mirror symmetry. The topological superconductivity is featured by 2Z degenerate Dirac cones on the (1,0) edge, and Z pairs of Majorana modes at the intersection between the (1,1) and (1,-1) edges. The topological invariance and Fermi surface criterion for the topological state are provided. Moreover, we point out that the previously proposed s-wave state in iron-based superconductors, which features a sign-changed superconducting order parameter between two electron pockets, is such a topological state. Thus, these results not only open a new route to pursue topological superconductivity, but also establish a measurable quantity to settle one long-lasting debate on the pairing nature of iron-based superconductors.