Differential cross sections for elastic Compton scattering from $^4$He have been measured with high statistical precision at the High Intensity $gamma$-ray Source at laboratory scattering angles of $55^circ$, $90^circ$, and $125^circ$ using a quasi-monoenergetic photon beam with a weighted mean energy value of $81.3$ MeV. The results are compared to previous measurements and similar fore-aft asymmetry in the angular distribution of the differential cross sections is observed. This experimental work is expected to strongly motivate the development of effective-field-theory calculations of Compton scattering from $^4$He to fully interpret the data.