Observations of the redshift z=7.085 quasar J1120+0641 have been used to search for variations of the fine structure constant, alpha, over the redshift range 5.5 to 7.1. Observations at z=7.1 probe the physics of the universe when it was only 0.8 billion years old. These are the most distant direct measurements of alpha to date and the first measurements made with a near-IR spectrograph. A new AI analysis method has been employed. Four measurements from the X-SHOOTER spectrograph on the European Southern Observatorys Very Large Telescope (VLT) directly constrain any changes in alpha relative to the value measured on Earth (alpha_0). The weighted mean strength of the electromagnetic force over this redshift range in this location in the universe is da/a = (alpha_z - alpha_0)/alpha_0 = (-2.18 +/- 7.27) X 10^{-5}, i.e. we find no evidence for a temporal change from the 4 new very high redshift measurements. When the 4 new measurements are combined with a large existing sample of lower redshift measurements, a new limit on possible spatial variation of da/a is marginally preferred over a no-variation model at the 3.7 sigma level.