Ultra-fast transmission electron microscopy (UTEM) combines sub-picosecond time-resolution with the versatility of TEM spectroscopies. It allows one to study the dynamics of materials properties combining complementary techniques. However, until now, time-resolved cathodoluminescence, which is expected to give access to the optical properties dynamics, was still unavailable in a UTEM. In this paper, we report time-resolved cathodoluminescence measurements in an ultrafast transmission electron microscope. We measured lifetime maps, with a 12 nm spatial resolution and sub-nanoseconds resolution, of nano-diamonds with a high density of NV center. This study paves the way to new applications of UTEM and to correlative studies of optically active nanostructures.