We have observed propagation of Edge Magneto-Plasmon (EMP) modes in graphene in the Quantum Hall regime by performing picosecond time of flight measurements between narrow contacts on the perimeter of micrometric exfoliated graphene. We find the propagation to be chiral with low attenuation and to have a velocity which is quantized on Hall plateaus. The velocity has two contributions, one arising from the Hall conductivity and the other from carrier drift along the edge, which we were able to separate by their different filling factor dependence. The drift component is found to be slightly less than the Fermi velocity as expected for graphene dynamics in an abrupt edge potential. The Hall conduction contribution is slower than expected and indicates a characteristic length in the Coulomb potential from the Hall charge of about 500 nm. The experiment illustrates how EMP can be coupled to the electromagnetic field, opening the perspective of GHz to THz chiral plasmonics applications to devices such as voltage controlled phase shifters, circulators, switches and compact, tunable ring resonators.