We study the phenomenon of discrete symmetry breaking during the inflationary epoch, using a model-independent approach based on the effective field theory of inflation. We work in a context where both time reparameterization symmetry and spatial diffeomorphism invariance can be broken during inflation. We determine the leading derivative operators in the quadratic action for fluctuations that break parity and time-reversal. Within suitable approximations, we study their consequences for the dynamics of linearized fluctuations. Both in the scalar and tensor sectors, we show that such operators can lead to new direction-dependent phases for the modes involved. They do not affect the power spectra, but can have consequences for higher correlation functions. Moreover, a small quadrupole contribution to the sound speed can be generated.