In this paper, we study the $sigma$-self-orthogonality of constacyclic codes of length $p^s$ over the finite commutative chain ring $mathbb F_{p^m} + u mathbb F_{p^m}$, where $u^2=0$ and $sigma$ is a ring automorphism of $mathbb F_{p^m} + u mathbb F_{p^m}$. First, we obtain the structure of $sigma$-dual code of a $lambda$-constacyclic code of length $p^s$ over $mathbb F_{p^m} + u mathbb F_{p^m}$. Then, the necessary and sufficient conditions for a $lambda$-constacyclic code to be $sigma$-self-orthogonal are provided. In particular, we determine the $sigma$-self-dual constacyclic codes of length $p^s$ over $mathbb F_{p^m} + u mathbb F_{p^m}$. Finally, we extend the results to constacyclic codes of length $2 p^s$.