Combining experiments with first principles calculations, we show that site-specific doping of Mn into SrTiO3 has a decisive influence on the dielectric properties of these doped systems. We find that phonon contributions to the dielectric constant invariably decrease sharply on doping at any site. However, a sizable, random dipolar contribution only for Mn at the Sr site arises from a strong off-centric displacement of Mn in spite of Mn being in a non-d0 state; this leads to a large dielectric constant at higher temperatures and gives rise to a relaxor ferroelectric behavior at lower temperatures. We also investigate magnetic properties in detail and critically reevaluate the possibility of a true multi-glass state in such systems.