We study the scattering of a pseudoscalar meson off one ground state octet baryon in covariant baryon chiral perturbation theory (BChPT) up to the next-to-next-to-leading order. The inherent power counting breaking terms are removed within extended-on-mass-shell scheme. We perform the first combined study of the pion-nucleon and kaon-nucleon scattering data in covariant BChPT and show that it can provide a reasonable description of the experimental data. In addition, we find that it is possible to fit the experimental baryon masses and the pion-nucleon and kaon-nucleon scattering data simultaneously at this order, thus providing a consistent check on covariant BChPT. We compare the scattering lengths of all the pertinent channels with available experimental data and those of other approaches. In addition, we have studied the leading order contributions of the virtual decuplet and found that they can improve the description of the $pi N$ phase shifts near the $Delta(1232)$ peak, while they have negligible effects on the description of the $K N$ phase shifts.