Complex oxide interfaces are a promising platform for studying a wide array of correlated electron phenomena in low-dimensions, including magnetism and superconductivity. The microscopic origin of these phenomena in complex oxide interfaces remains an open question. Here we investigate for the first time the magnetic properties of semi-insulating NdTiO$_3$/SrTiO$_3$ (NTO/STO) interfaces and present the first milli-Kelvin study of NTO/STO. The magnetoresistance (MR) reveals signatures of local ferromagnetic order and of spin-dependent thermally-activated transport, which are described quantitatively by a simple phenomenological model. We discuss possible origins of the interfacial ferromagnetism. In addition, the MR also shows transient hysteretic features on a timescale of ~10-100 seconds. We demonstrate that these are consistent with an extrinsic magneto-thermal origin, which may have been misinterpreted in previous reports of magnetism in STO-based oxide interfaces. The existence of these two MR regimes (steady-state and transient) highlights the importance of time-dependent measurements for distinguishing signatures of ferromagnetism from other effects that can produce hysteresis at low temperatures.