Atomic layer deposition (ALD) provides uniform and conformal thin films that are of interest for a range of applications. To better understand the properties of amorphous ALD films, we need improved understanding of their local atomic structure. Previous work demonstrated measurement of how the local atomic structure of ALD-grown aluminum oxide (AlOx) evolves in operando during growth by employing synchrotron high energy X-ray diffraction (HE-XRD). In this work, we report on efforts to employ electron diffraction pair distribution function (ePDF) measurements using more broadly available transmission electron microscope (TEM) instrumentation to study the atomic structure of amorphous ALD-AlOx. We observe electron beam damage in the ALD-coated samples during ePDF at ambient temperature and successfully mitigate this beam damage using ePDF at cryogenic temperatures (cryo-ePDF). We employ cryo-ePDF and Reverse Monte Carlo (RMC) modeling to obtain structural models of ALD-AlOx coatings formed at a range of deposition temperatures from 150-332{deg}C. From these model structures, we derive structural metrics including stoichiometry, pair distances, and coordination environments in the ALD-AlOx films as a function of deposition temperature. The structural variations we observe with growth temperature are consistent with temperature-dependent changes in the surface hydroxyl density on the growth surface. The sample preparation and cryo-ePDF procedures we report here can be used for routine measurement of ALD-grown amorphous thin films to improve our understanding of the atomic structure of these materials, establish structure-property relationships, and help accelerate the timescale for the application of ALD to address technological needs.