$gamma$-ray production cross sections have been measured in proton irradiations of N, Ne and Si and $alpha$-particle irradiations of N and Ne. In the same experiment we extracted also line shapes for strong $gamma$-ray lines of $^{16}$O produced in proton and $alpha$-particle irradiations of O. For the measurements gas targets were used for N, O and Ne and a thick foil was used for Si. All targets were of natural isotopic composition. Beams in the energy range up to 26 MeV for protons and 39 MeV for $alpha$-particles have been delivered by the IPN-Orsay tandem accelerator. The $gamma$ rays have been detected with four HP-Ge detectors in the angular range 30$^{circ}$ to 135$^{circ}$. We extracted 36 cross section excitation functions for proton reactions and 14 for $alpha$-particle reactions. For the majority of the excitation functions no other data exist to our knowledge. Where comparison with existing data was possible usually a very good agreement was found. It is shown that these data are very interesting for constraining nuclear reaction models. In particular the agreement of cross section calculations in the nuclear reaction code TALYS with the measured data could be improved by adjusting the coupling schemes of collective levels in the target nuclei $^{14}$N, $^{20,22}$Ne and $^{28}$Si. The importance of these results for the modeling of nuclear $gamma$-ray line emission in astrophysical sites is discussed.