We investigate the dependence of the low-mass X-ray binary (LMXB) population in early-type galaxies on stellar age, by selecting 20 massive nearby early-type galaxies from the Chandra archive occupying a relatively narrow range of masses and spanning a broad range of ages, from 1.6 Gyr to more than 10 Gyrs, with the median value of 6 Gyrs. With the ~ 2000 X-ray point sources detected in total, we correlated the specific number of LMXBs in each galaxy with its stellar age and globular cluster (GC) content. We found a correlation between the LMXB population and stellar age: older galaxies tend to possess about ~50% more LMXBs (per unit stellar mass) than the younger ones. The interpretation of this dependence is complicated by large scatter and a rather strong correlation between stellar age and GC content of galaxies in our sample. We present evidence suggesting that the more important factor may be the evolution of the LMXB population with time. Its effect is further amplified by the larger GC content of older galaxies and correspondingly, the larger numbers of dynamically formed binaries in them. We also found clear evolution of the X-ray luminosity function (XLF) with age, that younger galaxies have more bright sources and fewer faint sources per unit stellar mass. The XLF of LMXBs in younger galaxies appears to extend significantly beyond E39 erg/s. Such bright sources seem to be less frequent in older galaxies. We found that 6 out of ~ 12 (ultra-) luminous sources are located in GCs.