Charge dipole moment and spin moment rarely coexist in single-phase bulk materials except in some multiferroics. Despite the progress in the past decade, for most multiferroics their magnetoelectric performance remains poor due to the intrinsic exclusion between charge dipole and spin moment. As an alternative approach, the oxide heterostructures may evade the intrinsic limits in bulk materials and provide more attractive potential to realize the magnetoelectric functions. Here we perform a first-principles study on LaAlO$_3$/PbTiO$_3$ superlattices. Although neither of the components is magnetic, magnetic moments emerge at the ferroelectric domain walls of PbTiO$_3$ in these superlattices. Such a twist between ferroelectric domain and local magnetic moment, not only manifests an interesting type of multiferroicity, but also is possible useful to pursuit the electrical-control of magnetism in nanoscale heterostructures.