Detecting the imprint of inflationary gravitational waves on the $B$-mode polarization of the Cosmic Microwave Background (CMB) is one of the main science cases for current and next-generation CMB experiments. In this work we explore some of the challenges that ground-based facilities will have to face in order to carry out this measurement in the presence of Galactic foregrounds and correlated atmospheric noise. We present forecasts for Stage-3 (S3) and planned Stage-4 (S4) experiments based on the analysis of simulated sky maps using a map-based Bayesian foreground cleaning method. Our results thus consistently propagate the uncertainties on foreground parameters such as spatially-varying spectral indices, as well as the bias on the measured tensor-to-scalar ratio $r$ caused by an incorrect modelling of the foregrounds. We find that S3 and S4-like experiments should be able to put constraints on $r$ of the order $sigma(r)=(0.5-1.0)times10^{-2}$ and $sigma(r)=(0.5-1.0)times10^{-3}$ respectively, assuming instrumental systematic effects are under control. We further study deviations from the fiducial foreground model, finding that, while the effects of a second polarized dust component would be minimal on both S3 and S4, a 2% polarized anomalous dust emission (AME) component would be clearly detectable by Stage-4 experiments.