State of the art nanomechanical resonators present quality factors Q ~ 10^3 - 10^5, which are much lower than those that can be naively extrapolated from the behavior of micromechanical resonators. We analyze the dissipation mechanism that arises in nanomechanical beam-structures due to the tunneling of mesoscopic phonons between the beam and its supports (known as clamping losses). We derive the environmental force spectral density that determines the quantum Brownian motion of a given resonance. Our treatment is valid for low frequencies and provides the leading contribution in the aspect ratio. This yields fundamental limits for the Q-values which are described by simple scaling laws and are relevant for state of the art experimental structures. In this context, for resonant frequencies in the 0.1-1GHz range, while this dissipation mechanism can limit flexural resonators it is found to be negligible for torsional ones. In the case of structureless 3D supports the corresponding environmental spectral densities are Ohmic for flexural resonators and super-Ohmic for torsional ones, while for 2D slab supports they yield 1/f noise. Furthermore analogous results are established for the case of suspended semiconducting single-walled carbon nanotubes. Finally, we provide a general expression for the spectral density that allows to extend our treatment to other geometries and illustrate its use by applying it to a microtoroid. Our analysis is relevant for applications in high precision measurements and for the prospects of probing quantum effects in a macroscopic mechanical degree of freedom.