We argue that different formulations of hydrodynamics are related to uncertainties in the definitions of local thermodynamic and hydrodynamic variables. We show that this ambiguity can be resolved by viewing different formulations of hydrodynamics as particular gauge choices which lead to the same physical behavior of the system. Using the example of bulk viscosity, we show that Bemfica-Disconzi-Noronha-Kovtun (BDNK) and Israel-Stewart hydrodynamics are particular gauge choices of this type, related by a well-defined transformation of thermodynamic and hydrodynamic variables. We argue that this gauge ambiguity is necessary to ascertain the causality of stochastic hydrodynamic evolution and conjecture that it could explain the applicability of hydrodynamics outside its expected regime of validity since far from equilibrium and close to equilibrium may be related through transformations of this type.