The neutron-rich $^{122-131}$Sb isotopes were produced as fission fragments in the reaction $^{9}$Be($^{238}$U,~f) with 6.2 MeV/u beam energy. An unique setup, consisting of AGATA, VAMOS++ and EXOGAM detectors, was used which enabled the prompt-delayed gamma-ray ($gamma$) spectroscopy of fission fragments in the time range of 100 ns - 200 $mu$s. New isomers, prompt and delayed transitions were established in the even-A $^{122-130}$Sb isotopes. In the odd-A $^{123-131}$Sb isotopes, new prompt and delayed $gamma$-ray transitions were identified, in addition to the confirmation of the previously known isomers. The half-lives of the isomeric states and the $B(E2)$ transition probabilities of the observed transitions depopulating these isomers were extracted. The experimental data was compared with the theoretical results obtained in the framework of Large-Scale Shell-Model (LSSM) calculations in a restricted model space. Modifications of several components of the shell model interaction were introduced to obtain a consistent agreement with the excitation energies and the $B(E2)$ transition probabilities in neutron-rich Sn and Sb isotopes. The isomeric configurations in Sn and Sb were found to be relatively pure. Further, the calculations revealed that the presence of a single valence proton, mainly in the $g_{7/2}$ orbital in Sb isotopes, leads to significant mixing (due to the $ upi$ interaction) of: (i) the neutron seniorities ($upsilon_{ u}$) and (ii) the neutron angular momentum ($I_{ u}$). The above features have a weak impact on the excitation energies, but have an important impact on the $B(E2)$ transition probabilities. In addition, a constancy of the relative excitation energies irrespective of neutron seniority and neutron number in Sn and Sb was observed.