We use the Ly-$alpha$ Mass Association Scheme (LyMAS; Peirani et al. 2014) to predict cross-correlations at $z=2.5$ between dark matter halos and transmitted flux in the Ly-$alpha$ forest, and compare to cross-correlations measured for quasars and damped Ly-$alpha$ systems (DLAs) from the Baryon Oscillation Spectroscopic Survey (BOSS) by Font-Ribera et al. (2012, 2013). We calibrate LyMAS using Horizon-AGN hydrodynamical cosmological simulations of a $(100 h^{-1} mathrm{Mpc})^3$ comoving volume. We apply this calibration to a $(1 h^{-1} mathrm{Gpc})^3$ simulation realized with $2048^3$ dark matter particles. In the 100 $h^{-1}$ Mpc box, LyMAS reproduces the halo-flux correlations computed from the full hydrodynamic gas distribution very well. In the 1 $h^{-1}$ Gpc box, the amplitude of the large scale cross-correlation tracks the halo bias $b_h$ as expected. We provide empirical fitting functions that describe our numerical results. In the transverse separation bins used for the BOSS analyses, LyMAS cross-correlation predictions follow linear theory accurately down to small scales. Fitting the BOSS measurements requires inclusion of random velocity errors; we find best-fit RMS velocity errors of 399 km s$^{-1}$ and 252 km s$^{-1}$ for quasars and DLAs, respectively. We infer bias-weighted mean halo masses of $M_h/10^{12} h^{-1}M_odot=2.19^{+0.16}_{-0.15}$ and $0.69^{+0.16}_{-0.14}$ for the host halos of quasars and DLAs, with $sim 0.2$ dex systematic uncertainty associated with redshift evolution, IGM parameters, and selection of data fitting range.