We investigate how the motive of hyper-Kahler varieties is controlled by weight-2 (or surface-like) motives via tensor operations. In the first part, we study the Voevodsky motive of singular moduli spaces of semistable sheaves on K3 and abelian surfaces as well as the Chow motive of their crepant resolutions, when they exist. We show that these motives are in the tensor subcategory generated by the motive of the surface, provided that a crepant resolution exists. This extends a recent result of Bulles to the OGrady-10 situation. In the non-commutative setting, similar results are proved for the Chow motive of moduli spaces of stable objects of the K3 category of a cubic fourfold. As a consequence, we provide abundant examples of hyper-Kahler varieties of OGrady-10 deformation type satisfying the standard conjectures. In the second part, we study the Andr{e} motive of projective hyper-Kahler varieties. We attach to any such variety its defect group, an algebraic group which acts on the cohomology and measures the difference between the full motive and its weight-2 part. When the second Betti number is not 3, we show that the defect group is a natural complement of the Mumford--Tate group inside the motivic Galois group, and that it is deformation invariant. We prove the triviality of this group for all known examples of projective hyper-Kahler varieties, so that in each case the full motive is controlled by its weight-2 part. As applications, we show that for any variety motivated by a product of known hyper-Kahler varieties, all Hodge and Tate classes are motivated, the motivated Mumford--Tate conjecture holds, and the Andre motive is abelian. This last point completes a recent work of Soldatenkov and provides a different proof for some of his results.