We investigate the generation of gravitational waves due to the gravitational instability of primordial density perturbations in an early matter-dominated era which could be detectable by experiments such as LIGO and LISA. We use relativistic perturbation theory to give analytic estimates of the tensor perturbations generated at second order by linear density perturbations. We find that large enhancement factors with respect to the naive second-order estimate are possible due to the growth of density perturbations on sub-Hubble scales. However very large enhancement factors coincide with a breakdown of linear theory for density perturbations on small scales. To produce a primordial gravitational wave background that would be detectable with LIGO or LISA from density perturbations in the linear regime requires primordial comoving curvature perturbations on small scales of order 0.02 for Advanced LIGO or 0.005 for LISA, otherwise numerical calculations of the non-linear evolution on sub-Hubble scales are required.