Due to their extreme luminosities, gamma-ray bursts (GRBs) can be detected in hostile regions of galaxies, nearby and at very high redshift, making them important cosmological probes. The investigation of galaxies hosting long-duration GRBs (whose progenitor is a massive star) demonstrated their connection to star formation. Still, the link to the total galaxy population is controversial, mainly because of the small-number statistics: ~ 1,100 are the GRBs detected so far, ~ 280 those with measured redshift, and ~ 70 the hosts studied in detail. These are typically low-redshift (z < 1.5), low luminosity, metal poor, and star-forming galaxes. On the other hand, at 1.5< z <4, massive, metal rich and dusty, interacting galaxies are not uncommon. The most distant population (z > 4) is poorly explored, but the deep limits reached point towards very small and star-forming objects, similar to the low-z population. This `back to the future behavior is a natural consequence of the connection of long GRBs to star formation in young regions of the universe.