The pre-main-sequence evolution of low-mass stars and brown dwarfs is studied numerically starting from the formation of a protostellar/proto-brown dwarf seed and taking into account the mass accretion onto the central object during the initial several Myr of evolution. The stellar evolution was computed using the STELLAR evolution code developed by Yorke & Bodenheimer with recent modifications by Hosokawa et al. The mass accretion rates were taken from numerical hydrodynamics models of Vorobyov & Basu computing the circumstellar disk evolution starting from the gravitational collapse of pre-stellar cloud cores of various mass and angular momentum. The resulting stellar evolution tracks were compared with the isochrones and isomasses calculated using non-accreting models. We find that mass accretion in the initial several Myr of protostellar evolution can have a strong effect on the subsequent evolution of young stars and brown dwarfs. The disagreement between accreting and non-accreting models in terms of the total stellar luminosity L_st, stellar radius R_st and effective temperature T_eff depends on the thermal efficiency of accretion, i.e., on the fraction of accretion energy absorbed by the central object. The largest mismatch is found for the cold accretion case, in which essentially all accretion energy is radiated away. The relative deviations in L_st and R_st in this case can reach 50% for 1.0-Myr-old objects and remain notable even for 10-Myr-old objects. In the hot and hybrid accretion cases, in which a constant fraction of accretion energy is absorbed, the disagreement between accreting and non-accreting models becomes less pronounced, but still remains notable for 1.0-Myr-old objects. These disagreements may lead to the wrong age estimate for objects of (sub-)solar mass when using the isochrones based on non-accreting models (abridged).