Traditional macro-cell networks are experiencing an upsurge of data traffic, and small-cells are deployed to help offload the traffic from macro-cells. Given the massive deployment of small-cells in a macro-cell, the aggregate power consumption of small-cells (though being low individually) can be larger than that of the macro-cell. Compared to the macro-cell base station (MBS) whose power consumption increases significantly with its traffic load, the power consumption of a small-cell base station (SBS) is relatively flat and independent of its load. To reduce the total power consumption of the heterogeneous networks (HetNets), we dynamically change the operating states (on and off) of the SBSs, while keeping the MBS on to avoid any service failure outside active small-cells. First, we consider that the wireless users are uniformly distributed in the network, and propose an optimal location-based operation scheme by gradually turning off the SBSs closer to the MBS. We then extend the operation problem to a more general case where users are non-uniformly distributed in the network. Although this problem is NP-hard, we propose a joint location and user density based operation scheme to achieve near-optimum (with less than 1% performance loss in our simulations) in polynomial time.