The discovery of extended TeV emission around the Geminga and PSR B0656+14 pulsars, with properties consistent with free particle propagation in the interstellar medium (ISM), has sparked considerable discussion on the possible presence of such halos in other systems. Here we make an assessment of the current TeV source population associated with energetic pulsars, in terms of size and estimated energy density. Based on two alternative estimators we conclude that a large majority of the known TeV sources have emission originating in the zone energetically and dynamically dominated by the pulsar (i.e. the pulsar wind nebula), rather than from a halo of particles diffusing in to the ISM. Furthermore, whilst the number of established halos will surely increase in the future, we find that it is unlikely that such halos contribute significantly to the total TeV $gamma$-ray luminosity from electrons accelerated in PWN.