We explore the system size dependence of heavy-quark-QGP interaction by studying the heavy flavor meson suppression and elliptic flow in Pb-Pb, Xe-Xe, Ar-Ar and O-O collisions at the LHC. The space-time evolution of the QGP is simulated using a (3+1)-dimensional viscous hydrodynamic model, while the heavy-quark-QGP interaction is described by an improved Langevin approach that includes both collisional and radiative energy loss inside a thermal medium. Within this framework, we provides a reasonable description of the $D$ meson suppression and flow coefficients in Pb-Pb collisions, as well as predictions for both $D$ and $B$ meson observables in other collision systems yet to be measured. We find a clear hierarchy for the heavy meson suppression with respect to the size of the colliding nuclei, while their elliptic flow coefficient relies on both the system size and the geometric anisotropy of the QGP. Sizable suppression and flow are predicted for both $D$ and $B$ mesons in O-O collisions, which serve as a crucial bridge of jet quenching between large and small collision systems. Scaling behaviors between different collision systems are shown for the nuclear suppression factor as a function of the participant number ($N_mathrm{part}$), and for the $N_mathrm{part}^{1/3}$-rescaled elliptic flow coefficient as a function of the centrality class of nuclear collisions.