Transport and Langevin equations are employed to study hadronic medium effects on charmonium elliptic flows in heavy-ion collisions. In Pb-Pb collisions, the anisotropic energy density of the quark-gluon plasma (QGP) in the transverse plane is transformed into hadron momentum anisotropy after the phase transition. Charmonia with high transverse momentum $p_T$ are produced via the primordial hard process and undergo different degrees of dissociation along different paths in the QGP. They then scatter with light hadrons in the hadron phase. Both contributions to the charmonium elliptic flows are studied at moderate and high transverse momenta. The elliptic flows of the prompt $J/psi$ are found to be considerably enhanced at high transverse momentum when the charmonium diffusion coefficients in the hadronic medium are parametrized through the geometry scale approximation. This hadronic medium effect is negligible for quarkonia with larger mass such as bottomonia.