We present the application of a fast quasi-adiabatic continuous method to the measurement of specific heat at 4He temperatures, which can be used for the study of a wide range of materials. The technique can be performed in the same configuration used for the relaxation method, as the typical time constants between calorimetric cell and thermal sink at 4.2 K are chosen to be of the order of tau~30 s. The accuracy in the absolute values have been tested by comparing them to relaxation-method results obtained in the same samples (performed in situ using the same set-up), with a deviation between the absolute values < 3% in the whole temperature range. This new version of the continuous calorimetric method at low temperatures allows us to completely characterize and measure a sample within a few hours with a high density of data points, whereas when employing other methods we typically need a few days. An exhaustive study has been performed for reproducibility to be tested. In the present work, we have applied this method to two different substances: CeSb2, which exhibits three magnetic transitions at 15.5 K, 11.7 K and 9.5 K, and graphite, both highly-oriented pyrolytic graphite (HOPG) and natural crystals. Our results on these graphites are discussed in comparison with previous published data on different kinds of graphite samples.