We have used diffusion Monte Carlo (DMC) calculations to study the structural properties of magnesium hydride (MgH$_2$), including the pressure-volume equation of state, the cohesive energy and the enthalpy of formation from magnesium bulk and hydrogen gas. The calculations employ pseudopotentials and B-spline basis sets to expand the single particle orbitals used to construct the trial wavefunctions. Extensive tests on system size, time step, and other sources of errors, performed on periodically repeated systems of up to 1050 atoms, show that all these errors together can be reduced to below 10 meV per formula unit. We find excellent agreement with the experiments for the equilibrium volume of both the Mg and the MgH$_2$ crystals. The cohesive energy of the Mg crystal is found to be 1.51(1) eV, and agrees perfectly with the experimental value of 1.51 eV. The enthalpy of formation of MgH$_2$ from Mg bulk and H$_2$ gas is found to be $0.85 pm 0.01$ eV/formula unit, or $82 pm 1$ kJ/mole, which is off the experimental one of $76.1 pm 1$ kJ/mole only by 6 kJ/mole. This shows that DMC can almost achieve chemical accuracy (1 kcal/mole) on this system. Density functional theory errors are shown to be much larger, and depend strongly on the functional employed.