We have investigated the evolution of ferromagnetic order in the correlated metal URhGe (Curie temperature $T_{rm C} = $9.5 K) by chemical substitution of Ru, Co and Si. Polycrystalline samples URh$_{1-x}$Ru$_x$Ge ($x leq $0.6), URh$_{1-x}$Co$_x$Ge ($x leq $0.9) and URhGe$_{1-x}$Si$_x$ ($x leq $0.2) have been prepared and the magnetic properties have been investigated by magnetization and transport experiments. In the case of Ru doping, $T_{rm C}$ initially increases, but then decreases linearly as a function of $x$ and is completely suppressed for $x_{rm cr} approx 0.38$. The Curie temperature in the URh$_{1-x}$Co$_x$Ge series has a broad maximum $T_{rm C} = 20$ K near $x=0.6$ and then drops to 8 K for $x=0.9$. In the case of Si doping $T_{rm C}$ stays roughly constant. We conclude that the alloy systems URh$_{1-x}$Ru$_x$Ge and URh$_{1-x}$Co$_x$Ge are interesting candidates to study the ferromagnetic instability.