We measured high resolution Cu $L_3$ edge resonant inelastic x-ray scattering (RIXS) of the undoped cuprates La$_2$CuO$_4$, Sr$_2$CuO$_2$Cl$_2$, CaCuO$_2$ and NdBa$_2$Cu$_3$O$_6$. The dominant spectral features were assigned to $dd$ excitations and we extensively studied their polarization and scattering geometry dependence. In a pure ionic picture, we calculated the theoretical cross sections for those excitations and used them to fit the experimental data with excellent agreement. By doing so, we were able to determine the energy and symmetry of Cu-3$d$ states for the four systems with unprecedented accuracy and confidence. The values of the effective parameters could be obtained for the single ion crystal field model but not for a simple two-dimensional cluster model. The firm experimental assessment of $dd$ excitation energies carries important consequences for the physics of high $T_c$ superconductors. On one hand, having found that the minimum energy of orbital excitation is always $geq 1.4$ eV, i.e., well above the mid-infrared spectral range, leaves to magnetic excitations (up to 300 meV) a major role in Cooper pairing in cuprates. On the other hand, it has become possible to study quantitatively the effective influence of $dd$ excitations on the superconducting gap in cuprates.