Reflection measurements give access to the complex impedance of a material on a wide frequency range. This is of interest to study the dynamical properties of various materials, for instance disordered superconductors. However reflection measurements made at cryogenic temperature suffer from the difficulty to reliably subtract the circuit contribution. Here we report on the design and first tests of a setup able to precisely calibrate in situ the sample reflection, at 4.2 K and up to 2 GHz, by switching and measuring, during the same cool down, the sample and three calibration standards.