GdFe3(BO3)4 exhibits a structural phase transition at 156 K, antiferromagnetic order of the Fe3+ moments at 36 K followed by a spin reorientation phase transition at 9 K. The reorientation phase transition is studied through dielectric, magnetic and heat capacity measurements under the application of external magnetic fields of up to 7 kOe. The dielectric constant indicates the existence of two distinct anomalies at T_SR = 9 K that separate in temperature under external magnetic fields. The spin rotation phase transition is proven to be of the first-order nature through the magnetic analogue of the Clausius-Clapeyron equation. Magneto-dielectric effect of up to 1% is observed at 8 K and 7 kOe. The uniaxial magnetocaloric effect along the c axis is observed below the spin reorientation phase transition of 9 K.