Monolayer transition metal dichalcogenides (TMDs) exhibit high nonlinear optical (NLO) susceptibilities. Experiments on MoS$_2$ have indeed revealed very large second-order ($chi^{(2)}$) and third-order ($chi^{(3)}$) optical susceptibilities. However, third harmonic generation results of other layered TMDs has not been reported. Furthermore, the reported $chi^{(2)}$ and $chi^{(3)}$ of MoS$_2$ vary by several orders of magnitude, and a reliable quantitative comparison of optical nonlinearities across different TMDs has remained elusive. Here, we investigate second- and third-harmonic generation, and three-photon photoluminescence in TMDs. Specifically, we present an experimental study of $chi^{(2)}$, and $chi^{(3)}$ of four common TMD materials (ce{MoS2}, ce{MoSe2}, ce{WS2} and ce{WSe2}) by placing different TMD flakes in close proximity to each other on a common substrate, allowing their NLO properties to be accurately obtained from a single measurement. $chi^{(2)}$ and $chi^{(3)}$ of the four monolayer TMDs have been compared, indicating that they exhibit distinct NLO responses. We further present theoretical simulations of these susceptibilities in qualitative agreement with the measurements. Our comparative studies of the NLO responses of different two-dimensional layered materials allow us to select the best candidates for atomic-scale nonlinear photonic applications, such as frequency conversion and all-optical signal processing.