Context. Multidimensional hydrodynamic simulations of convection in stellar interiors are numerically challenging, especially for flows at low Mach numbers. Methods. We explore the benefits of using a low-Mach hydrodynamic flux solver and demonstrate its usability for simulations in the astrophysical context. The time-implicit Seven-League Hydro (SLH) code was used to perform multidimensional simulations of convective helium shell burning based on a 25 M$_odot$ star model. The results obtained with the low-Mach AUSM$^{+}$-up solver were compared to results when using its non low-Mach variant AUSM$_mathrm{B}^{+}$-up. We applied well-balancing of the gravitational source term to maintain the initial hydrostatic background stratification. The computational grids have resolutions ranging from $180 times 90^2$ to $810 times 540^2$ cells and the nuclear energy release was boosted by factors of $3 times 10^3$, $1 times 10^4$, and $3 times 10^4$ to study the dependence of the results on these parameters. Results. The boosted energy input results in convection at Mach numbers in the range of $10^{-2}$ to $10^{-3}$. Standard mixing-length theory (MLT) predicts convective velocities of about $1.6 times 10^{-4}$ if no boosting is applied. Simulations with AUSM$^{+}$-up show a Kolmogorov-like inertial range in the kinetic energy spectrum that extends further toward smaller scales compared with its non low-Mach variant. The kinetic energy dissipation of the AUSM$^{+}$-up solver already converges at a lower resolution compared to AUSM$^{+}_{mathrm{B}}$ -up. The extracted entrainment rates at the boundaries of the convection zone are well represented by the bulk Richardson entrainment law and the corresponding fitting parameters are in agreement with published results for carbon shell burning.