We have studied the electronic structure of bulk single crystals and epitaxial films of magnetite Fe$_3$O$_4$. Fe $2p$ core-level spectra show clear differences between hard x-ray (HAX-) and soft x-ray (SX-) photoemission spectroscopy (PES), indicative of surface effects. The bulk-sensitive spectra exhibit temperature ($T$)-dependent charge excitations across the Verwey transition at $T_V$=122 K, which is missing in the surface-sensitive spectra. An extended impurity Anderson model full-multiplet analysis reveals roles of the three distinct Fe-species (A-Fe$^{3+}$, B-Fe$^{2+}$, B-Fe$^{3+}$) below $T_V$ for the Fe $2p$ spectra, and its $T-$dependent evolution. The Fe $2p$ HAXPES spectra show a clear magnetic circular dichroism (MCD) in the metallic phase of magnetized 100-nm-thick films. The model calculations also reproduce the MCD and identify the magnetically distinct sites associated with the charge excitations. Valence band HAXPES shows finite density of states at $E_F$ for the polaronic metal with remnant order above $T_V$, and a clear gap formation below $T_V$. The results indicate that the Verwey transition is driven by changes in the strongly correlated and magnetically active B-Fe$^{2+}$ and B-Fe$^{3+}$ electronic states, consistent with resistivity and bulk-sensitive optical spectra.