We present a study of the scaling behavior of the R{e}nyi entanglement entropy (REE) in SU($N$) spin chain Hamiltonians, in which all the spins transform under the fundamental representation. These SU($N$) spin chains are known to be quantum critical and described by a well known Wess-Zumino-Witten (WZW) non-linear sigma model in the continuum limit. Numerical results from our lattice Hamiltonian are obtained using stochastic series expansion (SSE) quantum Monte Carlo for both closed and open boundary conditions. As expected for this 1D critical system, the REE shows a logarithmic dependence on the subsystem size with a prefector given by the central charge of the SU($N$) WZW model. We study in detail the sub-leading oscillatory terms in the REE under both periodic and open boundaries. Each oscillatory term is associated with a WZW field and decays as a power law with an exponent proportional to the scaling dimension of the corresponding field. We find that the use of periodic boundaries (where oscillations are less prominent) allows for a better estimate of the central charge, while using open boundaries allows for a better estimate of the scaling dimensions. For completeness we also present numerical data on the thermal R{e}nyi entropy which equally allows for extraction of the central charge.