The study of the optical continuum of radio galaxies shows that about 30% have a young stellar population component. Among them are the most far-IR bright radio galaxies. A further indication of the relatively gas rich environment of these galaxies (possibly related to the recent merger from which they originate) is the high fraction being detected in HI. Here we present recent results obtained from the study of neutral hydrogen (detected either in emission or absorption) in a group of starburst radio galaxies. In some objects, large-scale (tens of kpc) structures involving HI masses exceeding 10^9 M_sun are observed. In these cases, the HI can be used to study the origin and evolution of these systems and the timescales involved. In this respect, the parameters obtained from the study of the stellar populations and from the HI can be complementary. In other objects, very broad (> 1000 km/s), mostly blueshifted HI is detected in absorption. This result shows that, despite the extremely energetic phenomena occurring near an AGN - including the powerful radio jet - some of the outflowing gas remains, or becomes again, neutral. This can give new and important insights in the physical conditions of the gaseous medium around an AGN. The possible origin of the extreme kinematics is discussed.