Weakly pumped systems with approximate conservation laws can be efficiently described by a generalized Gibbs ensemble if the steady state of the system is unique. However, such a description can fail if there are multiple steady state solutions, for example, a bistability. In this case domains and domain walls may form. In one-dimensional (1D) systems any type of noise (thermal or non-thermal) will in general lead to a proliferation of such domains. We study this physics in a 1D spin chain with two approximate conservation laws, energy and the $z$-component of the total magnetization. A bistability in the magnetization is induced by the coupling to suitably chosen Lindblad operators. We analyze the theory for a weak coupling strength $epsilon$ to the non-equilibrium bath. In this limit, we argue that one can use hydrodynamic approximations which describe the system locally in terms of space- and time-dependent Lagrange parameters. Here noise terms enforce the creation of domains, where the typical width of a domain wall goes as $sim 1/sqrt{epsilon}$ while the density of domain walls is exponentially small in $1/sqrt{epsilon}$. This is shown by numerical simulations of a simplified hydrodynamic equation in the presence of noise.