We introduce a fundamental restriction on the strain energy function and stress tensor for initially stressed elastic solids. The restriction applies to strain energy functions $W$ that are explicit functions of the elastic deformation gradient $mathbf{F}$ and initial stress $boldsymbol tau$, i.e. $W:= W(mathbf F, boldsymbol tau)$. The restriction is a consequence of energy conservation and ensures that the predicted stress and strain energy do not depend upon an arbitrary choice of reference configuration. We call this restriction: initial stress reference independence (ISRI). It transpires that almost all strain energy functions found in the literature do not satisfy ISRI, and may therefore lead to unphysical behaviour, which we illustrate via a simple example. To remedy this shortcoming we derive three strain energy functions that do satisfy the restriction. We also show that using initial strain (often from a virtual configuration) to model initial stress leads to strain energy functions that automatically satisfy ISRI. Finally, we reach the following important result: ISRI reduces the number of unknowns of the linear stress tensor of initially stressed solids. This new way of reducing the linear stress may open new pathways for the non-destructive determination of initial stresses via ultrasonic experiments, among others.