First-principles studies of the electron-phonon coupling in graphene predict a high coupling strength for the $sigma$ band with $lambda$ values of up to 0.9. Near the top of the $sigma$ band, $lambda$ is found to be $approx 0.7$. This value is consistent with the recently observed kinks in the $sigma$ band dispersion by angle-resolved photoemission. While the photoemission intensity from the $sigma$ band is strongly influenced by matrix elements due to sub-lattice interference, these effects differ significantly for data taken in the first and neighboring Brillouin zones. This can be exploited to disentangle the influence of matrix elements and electron-phonon coupling. A rigorous analysis of the experimentally determined complex self-energy using Kramers-Kronig transformations further supports the assignment of the observed kinks to strong electron-phonon coupling and yields a coupling constant of $0.6(1)$, in excellent agreement with the calculations.