The perovskite TbFe$_{0.5}$Cr$_{0.5}$O$_3$ shows two anomalies in the magnetic susceptibility at $T_N$ = 257K and $T_{SR}$ = 190K which are respectively, the antiferromagnetic and spin reorientation transition that occur in the Fe/Cr sublattice. Analysis of the magnetic susceptibility reveals signatures of Griffiths-like phase in this compound. Neutron diffraction analysis confirms that, as the temperature is reduced from 350K, a spin reorientation transition from $Gamma_2$ (F$_x$, C$_y$, G$_z$) to $Gamma_4$ (G$_x$, A$_y$, F$_z$) occurs at $T_N$ = 257K and subsequently, a second spin reorientation takes place from $Gamma_4$ (G$_x$, A$_y$, F$_z$) to $Gamma_2$ (F$_x$, C$_y$, G$_z$) at $T_{SR}$ = 190K. The $Gamma_2$ (F$_x$, C$_y$, G$_z$) structure is stable until 7.7K where an ordered moment of 7.74(1)$mu_mathrm B$/Fe$^{3+}$(Cr$^{3+}$) is obtained from neutron data refinement. In addition to the long-range order of the magnetic structure, indication of diffuse magnetic scattering at 7.7K is evident, thereby lending support to the Griffiths-like phase observed in susceptibility. At 7.7K, Tb develops a ferromagnetic component along the crystallographic $a$ axis. Thermal conductivity, and spin-phonon coupling of TbFe$_{0.5}$Cr$_{0.5}$O$_3$ through Raman spectroscopy are studied in the present work. An antiferromagnetic structure with ($uparrow downarrow uparrow downarrow$) arrangement of Fe/Cr spins is found in the ground state through first-principles energy calculations which supports the experimental magnetic structure at 7.7K. The spin-resolved total and partial density of states are determined showing that TbFe$_{0.5}$Cr$_{0.5}$O$_3$ is insulating with a band gap of $sim 0.12$ (2.4) eV within GGA (GGA+$U$) functionals.