The low-temperature magnetic phase diagram of the multiferroic system FeTe$_2$O$_5$Br down to 300 mK and up to 9 T is presented. Short-range magnetic correlations within the crystal layers start to develop already at $sim$50 K, i.e., far above $T_{N1} sim$ 11.0 K, where the system undergoes a magnetic phase transition into the high-temperature incommensurate (HT-ICM) phase. Only 0.5 K lower, at $T_{N2}$, the system undergoes a second phase transition into the low-temperature incommensurate amplitude-modulated (LT-ICM) phase accompanied by a spontaneous electric polarization. When the magnetic field is applied, the transition temperatures shift depending on the field orientation. In the case of $B||b$ and $B >$ 4.5 T, the HT-ICM phase disappears along with the electric polarization in the LT-ICM phase. The field dependence of the magnetic transition temperatures is explained in the context of the magnetic susceptibility behavior. Similarities and differences between the novel amplitude-modulated and well-established helicoidal magnetoelectrics are discussed.