The pseudogap phenomenon in cuprates is the most mysterious puzzle in the research of high-temperature superconductivity. In particular, whether the pseudogap is associated with a crossover or phase transition has been a long-standing controversial issue. The tetragonal cuprate HgBa$_2$CuO$_{4+delta}$, with only one CuO$_2$ layer per primitive cell, is an ideal system to tackle this puzzle. Here, we measure the anisotropy of magnetic susceptibility within the CuO$_2$ plane with exceptionally high-precision magnetic torque experiments. Our key finding is that a distinct two-fold in-plane anisotropy sets in below the pseudogap temperature $T^*$, which provides thermodynamic evidence for a nematic phase transition with broken four-fold symmetry. Most surprisingly, the nematic director orients along the diagonal direction of the CuO$_2$ square lattice, in sharp contrast to the bond nematicity reported in various iron-based superconductors and double-layer YBa$_2$Cu$_3$O$_{6+delta}$, where the anisotropy axis is along the Fe-Fe and Cu-O-Cu directions, respectively. Another remarkable feature is that the enhancement of the diagonal nematicity with decreasing temperature is suppressed around the temperature at which short-range charge-density-wave (CDW) formation occurs. This is in stark contrast to YBa$_2$Cu$_3$O$_{6+delta}$, where the bond nematicity is not influenced by the CDW. Our result suggests a competing relationship between diagonal nematic and CDW order in HgBa$_2$CuO$_{4+delta}$.