Many natural and physical processes display long memory and extreme events. In these systems, the measured time series is invariably contaminated by noise. As the extreme events display large deviation from the mean behaviour, the noise does not affect the extreme events as much as it affects the typical values. Since the extreme events also carry the information about correlations in the full time series, they can be used to infer the correlation properties of the latter. In this work, from a given time series, we construct three modified time series using only the extreme events. It is shown that the correlations in the original time series and in the modified time series, as measured by the exponent obtained from detrended fluctuation analysis technique, are related to each other. Hence, the correlation exponents for a long memory time series can be inferred from its extreme events alone. This approach is demonstrated for several empirical time series.