The mathematics of gauge theories lies behind many of the most profound advances in physics in the last 200 years, from Maxwells theory of electromagnetism to Einsteins theory of general relativity. More recently it has become clear that gauge theories also emerge in condensed matter, a prime example being the spin ice materials which host an emergent electromagnetic gauge field. In spin ice, the underlying gauge structure is revealed by the presence of pinch-point singularities in neutron-scattering measurements. Here we report the discovery of a spin liquid where the low-temperature physics is naturally described by the fluctuations of a tensor field with a continuous gauge freedom. This gauge structure underpins an unusual form of spin correlations, giving rise to pinch-line singularities--- line-like analogues of the pinch-points observed in spin ice. Remarkably, these features may already have been observed in the pyrochlore material Tb$_2$Ti$_2$O$_7$.