This paper concerns the convex optimal control problem governed by multiscale elliptic equations with arbitrarily rough $L^infty$ coefficients, which has important applications in composite materials and geophysics. We use one of the recently developed numerical homogenization techniques, the so-called Rough Polyharmonic Splines (RPS) and its generalization (GRPS) for the efficient resolution of the elliptic operator on the coarse scale. Those methods have optimal convergence rate which do not rely on the regularity of the coefficients nor the concepts of scale-separation or ergodicity. As the iterative solution of the OCP-OPT formulation of the optimal control problem requires solving the corresponding (state and co-state) multiscale elliptic equations many times with different right hand sides, numerical homgogenization approach only requires one-time pre-computation on the fine scale and the following iterations can be done with computational cost proportional to coarse degrees of freedom. Numerical experiments are presented to validate the theoretical analysis.