We study cosmological perturbation theory within the framework of unimodular gravity. We show that the Lagrangian constraint on the determinant of the metric required by unimodular gravity leads to an extra constraint on the gauge freedom of the metric perturbations. Although the main equation of motion for the gravitational potential remains the same, the shift variable, which is gauge artifact in General Relativity, cannot be set to zero in unimodular gravity. This non-vanishing shift variable affects the propagation of photons throughout the cosmological evolution and therefore modifies the Sachs-Wolfe relation between the relativistic gravitational potential and the microwave temperature anisotropies. However, for adiabatic fluctuations the difference between the result in General Relativity and unimodular gravity is suppressed on large angular scales. Thus, no strong constraints on the theory can be derived.