We study the spin-orbit torque (SOT) effective fields in Cr/CoFeAl/MgO and Ru/CoFeAl/MgO magnetic heterostructures using the adiabatic harmonic Hall measurement. High-quality perpendicular-magnetic-anisotropy CoFeAl layers were grown on Cr and Ru layers. The magnitudes of the SOT effective fields were found to significantly depend on the underlayer material (Cr or Ru) as well as their thicknesses. The damping-like longitudinal effective field ({Delta}H_L) increases with increasing underlayer thickness for all heterostructures. In contrast, the field-like transverse effective field ({Delta}H_T) increases with increasing Ru thickness while it is almost constant or slightly decreases with increasing Cr thickness. The sign of {Delta}H_L observed in the Cr-underlayer devices is opposite from that in the Ru-underlayer devices while {Delta}H_T shows the same sign with a small magnitude. The opposite directions of {Delta}HL indicate that the signs of spin Hall angle in Cr and Ru are opposite, which are in good agreement with theoretical predictions. These results show sizable contribution from SOT even for elements with small spin orbit coupling such as 3d Cr and 4d Ru.