Improvements of the systematic uncertainty, frequency instability, and long-term reliability of the two caesium fountain primary frequency standards CSF1 and CSF2 at PTB (Physikalisch-Technische Bundesanstalt) are described. We have further investigated many of the systematic effects and made a number of modifications of the fountains. With an optically stabilized microwave oscillator, the quantum projection noise limited frequency instabilities are improved to $7.2 times 10^{-14} (tau/1,mathrm{s})^{-1/2}$ for CSF1 and $2.5 times 10^{-14} (tau/1,mathrm{s})^{-1/2}$ for CSF2 at high atom density. The systematic uncertainties of CSF1 and CSF2 are reduced to $2.74 times 10^{-16}$ and $1.71 times 10^{-16}$, respectively. Both fountain clocks regularly calibrate the scale unit of International Atomic Time (TAI) and the local realization of Coordinated Universal Time, UTC(PTB), and serve as references to measure the frequencies of local and remote optical frequency standards.