The intermediate valence (IV) compound YbAl$_3$ exhibits nonintegral valence (Yb 4$f^{14-n_f}$ (5d6s)$^z$ where z = 2+n$_f$ = 2.75) in a moderately heavy (m* = 20-30me) ground state with a large Kondo temperature (T$_K$ ~ 500-600K). We have measured the magnetic fluctuations and the phonon spectra on single crystals of this material by time-of-flight inelastic neutron scattering (INS) and inelastic x-ray scattering (IXS). We find that at low temperature, the Kondo-scale spin fluctuations have a momentum (Q) dependence similar to that seen recently in the IV compound CePd$_3$ and which can be attributed to particle-hole excitations in a coherent itinerant 4$f$ correlated ground state. The Q-dependence disappears as the temperature is raised towards room temperature and the 4$f$ electron band states become increasingly incoherent. The measured phonons can be described adequately by a calculation based on standard DFT+$U$ density functional theory, without recourse to considering 4$f$ correlations dynamically. A low temperature magnetic peak observed in the neutron scattering at ~ 30meV shows dispersion identical to an optic phonon branch. This 4$f$/phonon resonance disappears for T > 150K. The phonons appear to remain unaffected by the resonance. We discuss several possibilities for the origin of this unusual excitation, including the idea that it arises from the large amplitude beating of the light Al atoms against the heavy Yb atoms, resulting in a dynamic 4$f$/3$p$ hybridization.