A basic principle of long baseline interferometry is that an optical path difference (OPD) directly translates into an astrometric measurement. In the simplest case, the OPD is equal to the scalar product between the vector linking the two telescopes and the normalized vector pointing toward the star. However, a too simple interpretation of this scalar product leads to seemingly conflicting results, called here the baseline paradox. For micro-arcsecond accuracy astrometry, we have to model in full the metrology measurement. It involves a complex system subject to many optical effects: from pure baseline errors to static, quasi-static and high order optical aberrations. The goal of this paper is to present the strategy used by the General Relativity Analysis via VLT InTerferometrY instrument (GRAVITY) to minimize the biases introduced by these defects. It is possible to give an analytical formula on how the baselines and tip-tilt errors affect the astrometric measurement. This formula depends on the limit-points of three type of baselines: the wide-angle baseline, the narrow-angle baseline, and the imaging baseline. We also, numerically, include non-common path higher-order aberrations, whose amplitude were measured during technical time at the Very Large Telescope Interferometer. We end by simulating the influence of high-order common-path aberrations due to atmospheric residuals calculated from a Monte-Carlo simulation tool for Adaptive optics systems. The result of this work is an error budget of the biases caused by the multiple optical imperfections, including optical dispersion. We show that the beam stabilization through both focal and pupil tracking is crucial to the GRAVITY system. Assuming the instrument pupil is stabilized at a 4 cm level on M1, and a field tracking below 0.2$lambda/D$, we show that GRAVITY will be able to reach its objective of 10$mu$as accuracy.