Oxide heterostructures exhibit a rich variety of magnetic and transport properties which arise due to contact at an interface. This can lead to surprising effects that are very different from the bulk properties of the materials involved. We report the magnetic properties of bilayers of SrRuO3, a well known ferromagnet, and CaRuO3, which is nominally a paramagnet. We find intriguing features that are consistent with CaRuO3 developing dual magnetic character, with both a net moment as well as antiferromagnetic order. We argue the ordered SrRuO3 layer induces an undulating polarization profile in the conduction electrons of CaRuO3, by a mechanism akin to Friedel oscillations. At low temperatures, this oscillating polarization is inherited by rigid local moments within CaRuO3, leading to a robust exchange bias. We present ab initio simulations in support of this picture. Our results demonstrate a new ordering mechanism and throw light on the magnetic character of CaRuO3 .