We present a theory of the isotropic-nematic quantum phase transition in the composite Fermi liquid arising in half-filled Landau levels. We show that the quantum phase transition between the isotropic and the nematic phase is triggered by an attractive quadrupolar interaction between electrons, as in the case of conventional Fermi liquids. We derive the theory of the nematic state and of the phase transition. This theory is based on the flux attachment procedure which maps an electron liquid in half-filled Landau levels into the composite Fermi liquid close to a nematic transition. We show that the local fluctuations of the nematic order parameters act as an effective dynamical metric interplaying with the underlying Chern-Simons gauge fields associated with the flux attachment. Both the fluctuations of the Chern-Simons gauge field and the nematic order parameter can destroy the composite fermion quasiparticles and drive the system into a non-Fermi liquid state. The effective field theory for the isotropic-nematic phase transition has $z = 3$ dynamical exponent due to Landau damping effects. We show that there is a Berry phase type term which governs the effective dynamics of the nematic order parameter fluctuations, which can be interpreted as a non-universal Hall viscosity of the dynamical metric. We show that the effective field theory has a Wen-Zee-type term. Both terms originate from the time-reversal breaking fluctuation of the Chern-Simons gauge fields. We present a perturbative computation of the Hall viscosity and also show that this term is also obtained by a Ward identity. We show that the disclination of the nematic fluid, carries an electric charge. We show that a resonance observed in radio-frequency conductivity experiments can be interpreted as a Goldstone nematic mode gapped by lattice effects.