The Habitable zone Planet Finder (HPF) is a fiber fed precise radial velocity spectrograph at the 10 m Hobby Eberly Telescope (HET). Due to its fixed altitude design, the HET pupil changes appreciably across a track, leading to significant changes of the fiber far-field illumination. HPFs fiber scrambler is designed to suppress the impact of these illumination changes on the radial velocities -- but the residual impact on the radial velocity measurements has yet to be probed on sky. We use GJ 411, a bright early type (M2) M dwarf to probe the effects of far-field input trends due to these pupil variations on HPF radial velocities (RVs). These large changes ($sim$ 2x) in pupil area and centroid present a harsh test of HPFs far-field scrambling. Our results show that the RVs are effectively decoupled from these extreme far-field input changes due to pupil centroid offsets, attesting to the effectiveness of the scrambler design. This experiment allows us to test the impact of these changes with large pupil variation on-sky, something we would not easily be able to do at a conventional optical telescope. While the pupil and illumination changes expected at these other telescopes are small, scaling from our results enables us to estimate and bound these effects, and show that they are controllable even for the new and next generation of RV instruments in their quest to beat down instrumental noise sources towards the goal of a few cm/s.