In superfluid $^3$He-B externally pumped quantized spin-wave excitations or magnons spontaneously form a Bose-Einstein condensate in a 3-dimensional trap created with the order-parameter texture and a shallow minimum in the polarizing field. The condensation is manifested by coherent precession of the magnetization with a common frequency in a large volume. The trap shape is controlled by the profile of the applied magnetic field and by the condensate itself via the spin-orbit interaction. The trapping potential can be experimentally determined with the spectroscopy of the magnon levels in the trap. We have measured the decay of the ground state condensates after switching off the pumping in the temperature range $(0.14div 0.2)T_{mathrm{c}}$. Two contributions to the relaxation are identified: (1) spin-diffusion with the diffusion coefficient proportional to the density of thermal quasiparticles and (2) the approximately temperature-independent radiation damping caused by the losses in the NMR pick-up circuit. The measured dependence of the relaxation on the shape of the trapping potential is in a good agreement with our calculations based on the magnetic field profile and the magnon-modified texture. Our values for the spin diffusion coefficient at low temperatures agree with the theoretical prediction and earlier measurements at temperatures above $0.5T_{mathrm{c}}$.