We determine the emph{$L_infty$-algebra} that controls deformations of a relative Rota-Baxter Lie algebra and show that it is an extension of the dg Lie algebra controlling deformations of the underlying LieRep pair by the dg Lie algebra controlling deformations of the relative Rota-Baxter operator. Consequently, we define the {em cohomology} of relative Rota-Baxter Lie algebras and relate it to their infinitesimal deformations. A large class of relative Rota-Baxter Lie algebras is obtained from triangular Lie bialgebras and we construct a map between the corresponding deformation complexes. Next, the notion of a emph{homotopy} relative Rota-Baxter Lie algebra is introduced. We show that a class of homotopy relative Rota-Baxter Lie algebras is intimately related to emph{pre-Lie$_infty$-algebras}.