We present a study of the HD 165054 astrometric calibration field that has been periodically observed with the Gemini Planet Imager. HD 165054 is a bright star within Baades Window, a region of the galactic plane with relatively low extinction from interstellar dust. HD 165054 was selected as a calibrator target due to the high number density of stars within this region ($sim 3$ stars per square arcsecond with $H<22$), necessary because of the small field-of-view of the Gemini Planet Imager. Using nine epochs spanning over five years, we have fit a standard five-parameter astrometric model to the astrometry of seven background stars within close proximity to HD 165054 (angular separation $< 2$ arcsec). We achieved a proper motion precision of $sim 0.3$ mas/yr, and constrained the parallax of each star to be $lesssim 1$ mas. Our measured proper motions and parallax limits are consistent with the background stars being a part of the galactic bulge. Using these measurements we find no evidence of any systematic trend of either the plate scale or the north angle offset of GPI between 2014 and 2019. We compared our model describing the motions of the seven background stars to observations of the same field in 2014 and 2018 obtained with Keck/NIRC2, an instrument with an excellent astrometric calibration. We find that predicted position of the background sources is consistent with that measured by NIRC2, within the uncertainties of the calibration of the two instruments. In the future, we will use this field as a standard astrometric calibrator for the upgrade of GPI and potentially for other high-contrast imagers.