We report on measurements of the adiabatic second order elastic constants of the off-stoichiometric Ni$_{54}$Mn$_{23}$Al$_{23}$ single crystalline Heusler alloy. The variation in the temperature dependence of the elastic constants has been investigated across the magnetic transition and over a broad temperature range. Anomalies in the temperature behaviour of the elastic constants have been found in the vicinity of the magnetic phase transition. Measurements under applied magnetic field, both isothermal and variable temperature, show that the value of the elastic constants depends on magnetic order, thus giving evidence for magnetoelastic coupling in this alloy system.