A microscopic calculation and symmetry argument reveal superconductivity in the vicinity of parity-violating magnetic order. An augmented cluster magnetic multipole order in a crystal lacking local space inversion parity may break global inversion symmetry, and then, it is classified into an odd-parity multipole order. We investigate unconventional superconductivity induced by an odd-parity magnetic multipole fluctuation in a two-dimensional two-sublattice Hubbard model motivated by Sr$_2$IrO$_4$. We find that even-parity superconductivity is more significantly suppressed by a spin-orbit coupling than that in a globally noncentrosymmetric system. Consequently, two odd-parity superconducting states are stabilized by magnetic multipole fluctuations in a large spin-orbit coupling region. Both of them are identified as $Z_2$ topological superconducting states. The obtained gap function of inter-sublattice pairing shows a gapped/nodal structure protected by nonsymmorphic symmetry. Our finding implies a new family of odd-parity topological superconductors. Candidate materials are discussed.