We consider the scheduling and resource allocation problem in AP-initiated uplink OFDMA transmissions of IEEE 802.11ax networks. The uplink OFDMA resource allocation problem is known to be non-convex and difficult to solve in general. However, due to the special subcarrier allocation model of IEEE 802.11ax, the utility maximization problem involving the instantaneous rates of stations can be formulated as an assignment problem, and hence can be solved using the Hungarian method. In this paper, we address the more general problem of stochastic network utility maximization. Specifically, we maximize the utility of long-term average rates of stations subject to average rate and power constraints using Lyapunov optimization. The resulting resource allocation policies perform arbitrarily close to optimal and have polynomial time complexity. An important advantage of the proposed framework is that it can be used along with the target wake time mechanism of IEEE 802.11ax to provide guarantees on the average power consumption and/or achievable rates of stations whenever possible. Two key applications of such a design approach are power-constrained IoT networks and battery-powered sensor networks. We complement the theoretical study with computer simulations that evaluate our approach against other existing methods.