This paper studies the extremum seeking control (ESC) problem for a class of constrained nonlinear systems. Specifically, we focus on a family of constraints allowing to reformulate the original nonlinear system in the so-called input-output normal form. To steer the system to optimize a performance function without knowing its explicit form, we propose a novel numerical optimization-based extremum seeking control (NOESC) design consisting of a constrained numerical optimization method and an inversion based feedforward controller. In particular, a projected gradient descent algorithm is exploited to produce the state sequence to optimize the performance function, whereas a suitable boundary value problem accommodates the finite-time state transition between each two consecutive points of the state sequence. Compared to available NOESC methods, the proposed approach i) can explicitly deal with output constraints; ii) the performance function can consider a direct dependence on the states of the internal dynamics; iii) the internal dynamics do not have to be necessarily stable. The effectiveness of the proposed ESC scheme is shown through extensive numerical simulations.