We generalize the definition of an exact sequence of tensor categories due to Brugui`eres and Natale, and introduce a new notion of an exact sequence of (finite) tensor categories with respect to a module category. We give three definitions of this notion and show their equivalence. In particular, the Deligne tensor product of tensor categories gives rise to an exact sequence in our sense. We also show that the dual to an exact sequence in our sense is again an exact sequence. This generalizes the corresponding statement for exact sequences of Hopf algebras. Finally, we show that the middle term of an exact sequence is semisimple if so are the other two terms.