It is known by now that amorphous solids at zero temperature do not possess a nonlinear elasticity theory: besides the shear modulus which exists, all the higher order coefficients do not exist in the thermodynamic limit. Here we show that the same phenomenon persists up to temperatures comparable to the glass transition. The zero temperature mechanism due to the prevalence of dangerous plastic modes of the Hessian matrix is replaced by anomalous stress fluctuations that lead to the divergence of the variances of the higher order elastic coefficients. The conclusion is that in amorphous solids elasticity can never be decoupled from plasticity: the nonlinear response is very substantially plastic.