Single crystal neutron diffraction is used to investigate the magnetic and structural phase diagram of the electron doped superconductor Ba(Fe$_{1-x}$Co$_x$)$_2$As$_2$. Heat capacity and resistivity measurements have demonstrated that Co doping this system splits the combined antiferromagnetic and structural transition present in BaFe$_2$As$_2$ into two distinct transitions. For $x$=0.025, we find that the upper transition is between the high-temperature tetragonal and low-temperature orthorhombic structures with ($T_{mathrm{TO}}=99 pm 0.5$ K) and the antiferromagnetic transition occurs at $T_{mathrm{AF}}=93 pm 0.5$ K. We find that doping rapidly suppresses the antiferromagnetism, with antiferromagnetic order disappearing at $x approx 0.055$. However, there is a region of co-existence of antiferromagnetism and superconductivity. The effect of the antiferromagnetic transition can be seen in the temperature dependence of the structural Bragg peaks from both neutron scattering and x-ray diffraction. We infer from this that there is strong coupling between the antiferromagnetism and the crystal lattice.