Star forming dwarf galaxies (SFDGs) have a high gas content and low metallicities, reminiscent of the basic entities in hierarchical galaxy formation scenarios. In the young universe they probably also played a major role in the cosmic reionization. Their abundant presence in the local volume and their youthful character make them ideal objects for detailed studies of the initial stellar mass function (IMF), fundamental star formation processes and its feedback to the interstellar medium. Occasionally we witness SFDGs involved in extreme starbursts, giving rise to strongly elevated production of super star clusters and global superwinds, mechanisms yet to be explored in more detail. SFDGs is the initial state of all dwarf galaxies and the relation to the environment provides us with a key to how different types of dwarf galaxies are emerging. In this review we will put the emphasis on the exotic starburst phase, as it seems less important for present day galaxy evolution but perhaps fundamental in the initial phase of galaxy formation.