We derive an exact matrix product state representation of the Haldane-Rezayi state on both the cylinder and torus geometry. Our derivation is based on the description of the Haldane-Rezayi state as a correlator in a non-unitary logarithmic conformal field theory. This construction faithfully captures the ten degenerate ground states of this model state on the torus. Using the cylinder geometry, we probe the gapless nature of the phase by extracting the correlation length, which diverges in the thermodynamic limit. The numerically extracted topological entanglement entropies seem to only probe the Abelian part of the theory, which is reminiscent of the Gaffnian state, another model state deriving from a non-unitary conformal field theory.