LaCrGe$_3$ has attracted attention as a paradigm example of the avoidance of ferromagnetic (FM) quantum criticality in an itinerant magnet. By combining thermodynamic, transport, x-ray and neutron scattering as well as $mu$SR measurements, we refined the temperature-pressure phase diagram of LaCrGe$_3$. We provide thermodynamic evidence (i) for the first-order character of the FM transition when it is suppressed to low temperatures and (ii) for the formation of new phases at high pressures. From our microscopic data, we infer that short-range FM ordered clusters exist in these high-pressure phases. These results suggest that LaCrGe$_3$ is a rare example, which fills the gap between the two extreme limits of avoided FM quantum criticality in clean and strongly disordered metals.