Studying the response of materials to strain can elucidate subtle properties of electronic structure in strongly correlated materials. So far, mostly the relation between strain and resistivity, the so called elastoresistivity, has been investigated. The elastocaloric effect is a second rank tensor quantity describing the relation between entropy and strain. In contrast to the elastoresistivity, the elastocaloric effect is a thermodynamic quantity. Experimentally, elastocaloric effect measurements are demanding since the thermodynamic conditions during the measurement have to be well controlled. Here we present a technique to measure the elastocaloric effect under quasi adiabatic conditions. The technique is based on oscillating strain, which allows for increasing the frequency of the elastocaloric effect above the thermal relaxation rate of the sample. We apply the technique to Co-doped iron pnictide superconductors and show that the thermodynamic signatures of second order phase transitions in the elastocaloric effect closely follow those observed in calorimetry experiments. In contrast to the heat capacity, the electronic signatures in the elastocaloric effect are measured against a small phononic background even at high temperatures, establishing this technique as a powerful complimentary tool for extracting the entropy landscape proximate to a continuous phase transition.