Langevin models are frequently used to model various stochastic processes in different fields of natural and social sciences. They are adapted to measured data by estimation techniques such as maximum likelihood estimation, Markov chain Monte Carlo methods, or the non-parametric direct estimation method introduced by Friedrich et al. The latter has the distinction of being very effective in the context of large data sets. Due to their $delta$-correlated noise, standard Langevin models are limited to Markovian dynamics. A non-Markovian Langevin model can be formulated by introducing a hidden component that realizes correlated noise. For the estimation of such a partially observed diffusion a different version of the direct estimation method was introduced by Lehle et al. However, this procedure includes the limitation that the correlation length of the noise component is small compared to that of the measured component. In this work we propose another version of the direct estimation method that does not include this restriction. Via this method it is possible to deal with large data sets of a wider range of examples in an effective way. We discuss the abilities of the proposed procedure using several synthetic examples.