We study origin of Rashba spin-orbit interaction at SrTiO$_3$ surfaces and LaAlO$_3$/SrTiO$_3$ interfaces by considering the interplay between atomic spin-orbit coupling and inversion asymmetry at the surface or interface. We show that, in a simple tight-binding model involving 3d $t_{2g}$ bands of Ti ions, the induced spin-orbit coupling in the $d_{xz}$ and $d_{yz}$ bands is cubic in momentum whereas the spin-orbit interaction in the $d_{xy}$ band has linear momentum dependence. We also find that the spin-orbit interaction in one-dimensional channels at LaAlO$_3$/SrTiO$_3$ interfaces is linear in momentum for all bands. We discuss implications of our results for transport experiments on SrTiO$_3$ surfaces and LaAlO$_3$/SrTiO$_3$ interfaces. In particular, we analyze the effect of a given spin-orbit interaction term on magnetotransport of LaAlO$_3$/SrTiO$_3$ by calculating weak anti-localization corrections to the conductance and to universal conductance fluctuations.