A method for introducing the higher order terms in the potential expansion to study the continuous limits of the Toda hierarchy is proposed in this paper. The method ensures that the higher order terms are differential polynomials of the lower ones and can be continued to be performed indefinitly. By introducing the higher order terms, the fewer equations in the Toda hierarchy are needed in the so-called recombination method to recover the KdV hierarchy. It is shown that the Lax pairs, the Poisson tensors, and the Hamiltonians of the Toda hierarchy tend towards the corresponding ones of the KdV hierarchy in continuous limit.