We study the van der Waals interaction between Rydberg alkali-metal atoms with fine structure ($n^2L_j$; $Lleq 2$) and heteronuclear alkali-metal dimers in the ground rovibrational state ($X^1Sigma^+$; $v=0$, $J=0$). We compute the associated $C_6$ dispersion coefficients of atom-molecule pairs involving $^{133}$Cs and $^{85}$Rb atoms interacting with KRb, LiCs, LiRb, and RbCs molecules. The obtained dispersion coefficients can be accurately fitted to a state-dependent polynomial $O(n^7)$ over the range of principal quantum numbers $40leq nleq 150$. For all atom-molecule pairs considered, Rydberg states $n^2S_j$ and $n^2P_j$ result in attractive $1/R^6$ potentials. In contrast, $n^2D_j$ states can give rise to repulsive potentials for specific atom-molecule pairs. The interaction energy at the LeRoy distance approximately scales as $n^{-5}$ for $n>40$. For intermediate values of $nlesssim40$, both repulsive and attractive interaction energies in the order of $ 10-100 ,mu$K can be achieved with specific atomic and molecular species. The accuracy of the reported $C_6$ coefficients is limited by the quality of the atomic quantum defects, with relative errors $Delta C_6/C_6$ estimated to be no greater than 1% on average.