An essential property of magnetic devices is the relaxation rate in magnetic switching which depends strongly on the damping in the magnetisation dynamics. It was recently measured that damping depends on the magnetic texture and, consequently, is a non-local quantity. The damping enters the Landau-Lifshitz-Gilbert equation as the phenomenological Gilbert damping parameter $alpha$, that does not, in a straight forward formulation, account for non-locality. Efforts were spent recently to obtain Gilbert damping from first principles for magnons of wave vector $mathbf{q}$. However, to the best of our knowledge, there is no report about real space non-local Gilbert damping $alpha_{ij}$. Here, a torque-torque correlation model based on a tight binding approach is applied to the bulk elemental itinerant magnets and it predicts significant off-site Gilbert damping contributions, that could be also negative. Supported by atomistic magnetisation dynamics simulations we reveal the importance of the non-local Gilbert damping in atomistic magnetisation dynamics. This study gives a deeper understanding of the dynamics of the magnetic moments and dissipation processes in real magnetic materials. Ways of manipulating non-local damping are explored, either by temperature, materials doping or strain.