Rhythmic activity in the gamma band (30-100Hz) has been observed in numerous animal species ranging from insects to humans, and in relation to a wide range of cognitive tasks. Various experimental and theoretical studies have investigated this rhythmic activity. The theoretical efforts have mainly been focused on the neuronal dynamics, under the assumption that network connectivity satisfies certain fine-tuning conditions required to generate gamma oscillations. However, it remains unclear how this fine tuning is achieved. Here we investigated the hypothesis that spike timing dependent plasticity (STDP) can provide the underlying mechanism for tuning synaptic connectivity to generate rhythmic activity in the gamma band. We addressed this question in a modeling study. We examined STDP dynamics in the framework of a network of excitatory and inhibitory neuronal populations that has been suggested to underlie the generation of gamma. Mean field Fokker Planck equations for the synaptic weights dynamics are derived in the limit of slow learning. We drew on this approximation to determine which types of STDP rules drive the system to exhibit gamma oscillations, and demonstrate how the parameters that characterize the plasticity rule govern the rhythmic activity. Finally, we propose a novel mechanism that can ensure the robustness of self-developing processes, in general and for rhythmogenesis in particular.