We present the direct detection of molecular hydrogen at the highest redshift known today (z_abs=4.224) in a Damped Lyman-alpha (DLA) system toward the quasar PSS J1443+2724. This absorber is remarkable for having one of the highest metallicities amongst DLA systems at z_abs>3, with a measured iron abundance relative to Solar of -1.12+/-0.10. We provide for the first time in this system accurate measurements of NI, MgII, SII and ArI column densities. The sulfur and nitrogen abundances relative to Solar, -0.63+/-0.10 and -1.38+/-0.10 respectively, correspond exactly to the primary nitrogen production plateau. H2 absorption lines are detected in four different rotational levels (J=0, 1, 2 and 3) of the vibrational ground-state in three velocity components with total column densities of log N(H2)=17.67, 17.97, 17.48 and 17.26 respectively. The J=4 level is tentatively detected in the strongest component with log N(H2)~14. The mean molecular fraction is log f=-2.38+/-0.13, with f=2N(H2)/(2N(H2)+N(HI)). We also measure log N(HD)/N(H2)<-4.2. The excitation temperatures T_{01} for the two main components of the system are 96 and 136 K respectively. We argue that the absorbing galaxy, whose star-formation activity must have started at least 2-5x10^8 yrs before z=4.224, is in a quiescent state at the time of observation. The density of the gas is small, n_H<=50 cm^{-3}, and the temperature is of the order of T~90-180 K. The high excitation of neutral carbon in one of the components can be explained if the temperature of the Cosmic Microwave Background Radiation has the value expected at the absorber redshift, T=14.2 K.