We present the noise performance of High Electron Mobility Transistors (HEMT) developed by CNRS-C2N laboratory. Various HEMTs gate geometries with 2 pF to 230 pF input capacitance have been studied at 4 K. A model for both voltage and current noises has been developed with frequency dependence up to 1 MHz. These HEMTs exhibit low dissipation, excellent noise performance and can advantageously replace traditional Si-JFETs for the readout of high impedance thermal sensor and semiconductor ionization cryogenic detectors. Our model predicts that cryogenic germanium detectors of 30 g with 10 eV heat and 20 eVee baseline resolution are feasible if read out by HEMT based amplifiers. Such resolution allows for high discrimination between nuclear and electron recoils at low threshold. This capability is of major interest for Coherent Elastic Neutrino Scattering and low-mass dark matter experiments such as Ricochet and EDELWEISS.