This paper is concerned with the numerical solution of the unified first order hyperbolic formulation of continuum mechanics recently proposed by Peshkov & Romenski, denoted as HPR model. In that framework, the viscous stresses are computed from the so-called distortion tensor A, which is one of the primary state variables. A very important key feature of the model is its ability to describe at the same time the behavior of inviscid and viscous compressible Newtonian and non-Newtonian fluids with heat conduction, as well as the behavior of elastic and visco-plastic solids. This is achieved via a stiff source term that accounts for strain relaxation in the evolution equations of A. Also heat conduction is included via a first order hyperbolic evolution equation of the thermal impulse, from which the heat flux is computed. The governing PDE system is hyperbolic and fully consistent with the principles of thermodynamics. It is also fundamentally different from first order Maxwell-Cattaneo-type relaxation models based on extended irreversible thermodynamics. The connection between the HPR model and the classical hyperbolic-parabolic Navier-Stokes-Fourier theory is established via a formal asymptotic analysis in the stiff relaxation limit. From a numerical point of view, the governing partial differential equations are very challenging, since they form a large nonlinear hyperbolic PDE system that includes stiff source terms and non-conservative products. We apply the successful family of one-step ADER-WENO finite volume and ADER discontinuous Galerkin finite element schemes in the stiff relaxation limit, and compare the numerical results with exact or numerical reference solutions obtained for the Euler and Navier-Stokes equations. To show the universality of the model, the paper is rounded-off with an application to wave propagation in elastic solids.