The traditional theory of magnetic moments for chiral phonons is based on the picture of the circular motion of the Born effective charge, typically yielding a small fractional value of the nuclear magneton. Here we investigate the adiabatic evolution of electronic states induced by lattice vibration of a chiral phonon and obtain an electronic orbital magnetization in the form of a topological second Chern form. We find that the traditional theory needs to be refined by introducing a $bm{k}$ resolved Born effective charge, and identify another contribution from the phonon-modified electronic energy together with the momentum-space Berry curvature. The second Chern form can diverge when there is a Yangs monopole near the parameter space of interest as illustrated by considering a phonon at the Brillouin zone corner in a gaped graphene model. We also find large magnetic moments for the optical phonon in bulk topological materials where non-topological contribution is also important. The magnetic moment experiences a sign change when the band inversion happens.