The intensity levels allowed by safety standards (ANSI or ICNIRP) limit the amount of light that can be used in a clinical setting to image highly scattering or absorptive tissues with Optical Coherence Tomography (OCT). To achieve high-sensitivity imaging at low intensity levels, we adapt a detection scheme -- which is used in quantum optics for providing information about spectral correlations of photons -- into a standard spectral domain OCT system. This detection scheme is based on the concept of Dispersive Fourier Transformation, where a fibre introduces a wavelength-dependent time delay measured by a single-pixel detector, usually a high-speed photoreceiver. Here, we use a fast Superconducting Single-Photon Detector (SSPD) as a single-pixel detector and obtain images of a glass stack and a slice of onion at the intensity levels of the order of 10 pW. We also provide a formula for a depth-dependent sensitivity fall-off in such a detection scheme which can be treated as a temporal equivalent of diffraction-grating-based spectrometers.