Recent technological advances allowed the coherent optical manipulation of high-energy electron wavepackets with attosecond precision. Here we theoretically investigate the collision of optically-modulated pulsed electron beams with atomic targets and reveal a quantum interference associated with different momentum components of the incident broadband electron pulse, which coherently modulates both the elastic and inelastic scattering cross sections. We show that the quantum interference has a high spatial sensitivity at the level of Angstroms, offering potential applications in high-resolution ultrafast electron microscopy. Our findings are rationalized by a simple model.