Beam-target double spin asymmetries and target single-spin asymmetries in exclusive $pi^+$ and $pi^-$ electroproduction were obtained from scattering of 1.6 to 5.7 GeV longitudinally polarized electrons from longitudinally polarized protons (for $pi^+$) and deuterons (for $pi^-$) using the CEBAF Large Acceptance Spectrometer (CLAS) at Jefferson Lab. The kinematic range covered is $1.1<W<2.6$ GeV and $0.05<Q^2<5$ GeV$^2$, with good anglular coverage in the forward hemisphere. The asymmetry results were divided into approximately 40,000 kinematic bins for $pi^+$ from free protons and 15,000 bins for $pi^-$ production from bound nucleons in the deuteron. The present results are found to be in reasonable agreement with fits to previous world data for $W<1.7$ GeV and $Q^2<0.5$ GeV$^2$, with discrepancies increasing at higher values of $Q^2$, especially for $W>1.5$ GeV. Very large target-spin asymmetries are observed for $W>1.6$ GeV. When combined with cross section measurements, the present results will provide powerful constraints on nucleon resonance amplitudes at moderate and large values of $Q^2$, for resonances with masses as high as 2.3 GeV.