The Hf-O system has been modeled by combining existing experimental data and first-principles calculations results through the CALPHAD approach. Special quasirandom structures of $alpha$ and $beta$ hafnium were generated to calculate the mixing behavior of oxygen and vacancies. For the total energy of oxygen, vibrational, rotational and translational degrees of freedom were considered. The Hf-O system was combined with previously modeled Hf-Si and Si-O systems, and the ternary compound in the Hf-Si-O system, HfSiO$_4$ has been introduced to calculate the stability diagrams pertinent to the thin film processing.