A mixed system of cooled and trapped, ions and atoms, paves the way for ion assisted cold chemistry and novel many body studies. Due to the different individual trapping mechanisms, trapped atoms are significantly colder than trapped ions, therefore in the combined system, the strong binary ion$-$atom interaction results in heat flow from ions to atoms. Conversely, trapped ions can also get collisionally heated by the cold atoms, making the resulting equilibrium between ions and atoms intriguing. Here we experimentally demonstrate, Rubidium ions (Rb$^+$) cool in contact with magneto-optically trapped (MOT) Rb atoms, contrary to the general expectation of ion heating for equal ion and atom masses. The cooling mechanism is explained theoretically and substantiated with numerical simulations. The importance of resonant charge exchange (RCx) collisions, which allows swap cooling of ions with atoms, wherein a single glancing collision event brings a fast ion to rest, is discussed.