We investigate the use of matrix product states (MPS) to approximate ground states of critical quantum spin chains with periodic boundary conditions (PBC). We identify two regimes in the (N,D) parameter plane, where N is the size of the spin chain and D is the dimension of the MPS matrices. In the first regime MPS can be used to perform finite size scaling (FSS). In the complementary regime the MPS simulations show instead the clear signature of finite entanglement scaling (FES). In the thermodynamic limit (or large N limit), only MPS in the FSS regime maintain a finite overlap with the exact ground state. This observation has implications on how to correctly perform FSS with MPS, as well as on the performance of recent MPS algorithms for systems with PBC. It also gives clear evidence that critical models can actually be simulated very well with MPS by using the right scaling relations; in the appendix, we give an alternative derivation of the result of Pollmann et al. [Phys. Rev. Lett. 102, 255701 (2009)] relating the bond dimension of the MPS to an effective correlation length.