The rapidly developing quantum technologies have put forward a requirement to precisely control and measure temperature of microscopic matters at quantum level. Many quantum thermometry schemes have been proposed. However, precisely measuring low temperature is still extremely challenging because the sensing errors obtained in these schemes tend to divergence with decreasing temperature. Using a continuous-variable system as a thermometer, we propose a non-Markovian quantum thermometry to measure the temperature of a quantum reservoir. A mechanism to make the sensing error $delta T$ scale with the temperature $T$ as $delta Tsimeq T$ in the full-temperature regime is discovered. Our analysis reveals that it is the quantum criticality of the total thermometer-reservoir system that causes this enhanced sensitivity. Solving the long-standing and challenging error-divergence problem, our result gives an efficient way to precisely measure the low temperature of quantum systems.