We study the dynamics of the quasi-one-dimensional Ising-Heisenberg antiferromagnet BaCo2V2O8 under a transverse magnetic field. Combining inelastic neutron scattering experiments and theoretical analyses by field theories and numerical simulations, we mainly elucidate the structure of the spin excitation spectrum in the high field phase, appearing above the quantum phase transition point mu0Hc ~ 10 T. We find that it is characterized by collective solitonic excitations superimposed on a continuum. These solitons are strongly bound in pairs due to the effective staggered field induced by the nondiagonal g tensor of the compound, and are topologically different from the fractionalized spinons in the weak field region. The dynamical susceptibility numerically calculated with the infinite time-evolving block decimation method shows an excellent agreement with the measured spectra, which enables us to identify the dispersion branches with elementary excitations. The lowest energy dispersion has an incommensurate nature and has a local minimum at an irrational wave number due to the applied transverse field.