Background: Common spatial pattern (CSP) has been widely used for feature extraction in the case of motor imagery (MI) electroencephalogram (EEG) recordings and in MI classification of brain-computer interface (BCI) applications. BCI usually requires relatively long EEG data for reliable classifier training. More specifically, before using general spatial patterns for feature extraction, a training dictionary from two different classes is used to construct a compound dictionary matrix, and the representation of the test samples in the filter band is estimated as a linear combination of the columns in the dictionary matrix. New method: To alleviate the problem of sparse small sample (SS) between frequency bands. We propose a novel sparse group filter bank model (SGFB) for motor imagery in BCI system. Results: We perform a task by representing residuals based on the categories corresponding to the non-zero correlation coefficients. Besides, we also perform joint sparse optimization with constrained filter bands in three different time windows to extract robust CSP features in a multi-task learning framework. To verify the effectiveness of our model, we conduct an experiment on the public EEG dataset of BCI competition to compare it with other competitive methods. Comparison with existing methods: Decent classification performance for different subbands confirms that our algorithm is a promising candidate for improving MI-based BCI performance.