Spin-orbit torques in ferromagnetic (FM)/non-magnetic (NM) heterostructures offer more energy-efficient means to realize spin-logic devices; however, their strengths are determined by the heterostructure interface. This work examines crystal orientation impact on the spin-orbit torque efficiency in different Fe/Pd bilayer systems. Spin torque ferromagnetic measurements evidence that the damping-like torque efficiency is higher in epitaxial than in polycrystalline bilayer structures while the field-like torque is negligible in all bilayer structures. The strength of the damping-like torque decreases with deterioration of the bilayer epitaxial quality. The present finding provides fresh insight for the enhancement of spin-orbit torques in magnetic heterostructures.