The van der Waals ferromagnet Fe$_3$GeTe$_2$ has recently attracted extensive research attention due to its intertwined magnetic, electronic and topological properties. Here, using high-resolution angle-resolved photoemission spectroscopy, we systematically investigate the temperature evolution of the electronic structure of bulk Fe$_3$GeTe$_2$. We observe largely dispersive energy bands that are narrowed by a factor of 1.6 compared with ab-initio calculation. Upon heating towards the ferromagnetic transition near 225 K, we observe a massive reduction of quasiparticle coherence in a large energy range, which is attributed to the enhanced magnetic fluctuation in the system. Remarkably, the electron bands barely shift with increasing temperature, which deviates from the exchange splitting picture within the itinerant Stoner model. We argue that the local magnetic moments play a crucial role in the ferromagnetism of Fe$_3$GeTe$_2$, despite its strongly itinerant nature. Our results provide important insights into the electronic and magnetic properties of Fe$_3$GeTe$_2$ and shed light on the generic understanding of itinerant magnetism in correlated materials.