NonGaussian cosmic shear statistics based on weak-lensing aperture mass ($M_{rm ap}$) maps can outperform the classical shear two-point correlation function ($gamma$-2PCF) in terms of cosmological constraining power. However, reaching the full potential of these new estimators requires accurate modeling of the physics of baryons as the extra nonGaussian information mostly resides at small scales. We present one such modeling based on the Magneticum hydrodynamical simulation for the KiDS-450 and DES-Y1 surveys and a Euclid-like survey. We compute the bias due to baryons on the lensing PDF and the distribution of peaks and voids in $M_{rm ap}$ maps and propagate it to the cosmological forecasts on the structure growth parameter $S_8$, the matter density parameter $Omega_{rm m}$, and the dark energy equation of state $w_0$ using the SLICS and cosmo-SLICS sets of dark-matter-only simulations. We report a negative bias of a few percent on $S_8$ and $Omega_{rm m}$ and also measure a positive bias of the same level on $w_0$ when including a tomographic decomposition. These biases reach $sim 5$% when combining $M_{rm ap}$ statistics with the $gamma$-2PCF as these estimators show similar dependency on the AGN feedback. We verify that these biases constitute a less than $1sigma$ shift on the probed cosmological parameters for current cosmic shear surveys. However, baryons need to be accounted for at the percentage level for future Stage IV surveys and we propose to include the uncertainty on the AGN feedback amplitude by marginalizing over this parameter using multiple simulations such as those presented in this paper. Finally, we explore the possibility of mitigating the impact of baryons by filtering the $M_{rm ap}$ map but find that this process would require to suppress the small-scale information to a point where the constraints would no longer be competitive.