The classical limit of polymer quantum theories yields a one parameter family of `effective theories labeled by lambda. Here we consider such families for constrained theories and pose the problem of taking the `continuum limit, lambda -> 0. We put forward criteria for such question to be well posed, and propose a concrete strategy based in the definition of appropriately constructed Dirac observables. We analyze two models in detail, namely a constrained oscillator and a cosmological model arising from loop quantum cosmology. For both these models we show that the program can indeed be completed, provided one makes a particular choice of lambda-dependent internal time with respect to which the dynamics is described and compared. We show that the limiting theories exist and discuss the corresponding limit. These results might shed some light in the problem of defining a renormalization group approach, and its associated continuum limit, for quantum constrained systems.