Discrete symmetries in grand canonical ensembles and in ensembles canonical with respect to triality are investigated. We speculate about the general phase structure of finite temperature gauge theories with discrete $Z(N)$ symmetry. Low and high temperature phases turn out to be different in both ensembles even for infinite systems. It is argued that gauge theories with matter fields in the fundamental representation should be treated in ensembles canonical with respect to triality if one wants to avoid unphysical predictions. Further, we discuss as a physical consequence of such a treatment the impossibility of the existence of metastable phases in the quark-gluon plasma.