To date, PMN J2134-0419 (at a redshift z=4.33) is the second most distant quasar known with a milliarcsecond-scale morphology permitting direct estimates of the jet proper motion. Based on two-epoch observations, we constrained its radio jet proper motion using the very long baseline interferometry (VLBI) technique. The observations were conducted with the European VLBI Network (EVN) at 5 GHz on 1999 November 26 and 2015 October 6. We imaged the central 10-pc scale radio jet emission and modeled its brightness distribution. By identifying a jet component at both epochs separated by 15.86 yr, a proper motion of mu=0.035 +- 0.023 mas/yr is found. It corresponds to an apparent superluminal speed of beta_a=4.1 +- 2.7 c . Relativistic beaming at both epochs suggests that the jet viewing angle with respect to the line of sight is smaller than 20 deg, with a minimum bulk Lorentz factor Gamma=4.3. The small value of the proper motion is in good agreement with the expectations from the cosmological interpretation of the redshift and the current cosmological model. Additionally we analyzed archival Very Large Array observations of J2143-0419 and found indication of a bent jet extending to ~30 kpc.