We present the activities carried out to calibrate and characterise the performance of the elements of attitude control and measurement on board the Herschel spacecraft. The main calibration parameters and the evolution of the indicators of the pointing performance are described, from the initial values derived from the observations carried out in the performance verification phase to those attained in the last year and half of mission, an absolute pointing error around or even below 1 arcsec, a spatial relative pointing error of some 1 arcsec and a pointing stability below 0.2 arsec. The actions carried out at the ground segment to improve the spacecraft pointing measurements are outlined. On-going and future developments towards a final refinement of the Herschel astrometry are also summarised. A brief description of the different components of the attitude control and measurement system (both in the space and in the ground segments) is also given for reference. We stress the importance of the cooperation between the different actors (scientists, flight dynamics and systems engineers, attitude control and measurement hardware designers, star-tracker manufacturers, etc.) to attain the final level of performance.