Clustering $unicode{x2013}$ the tendency for neighbors of nodes to be connected $unicode{x2013}$ quantifies the coupling of a complex network to its underlying latent metric space. In random geometric graphs, clustering undergoes a continuous phase transition, separating a phase with finite clustering from a regime where clustering vanishes in the thermodynamic limit. We prove this geometric-to-nongeometric phase transition to be topological in nature, with atypical features such as diverging free energy and entropy as well as anomalous finite size scaling behavior. Moreover, a slow decay of clustering in the nongeometric phase implies that some real networks with relatively high levels of clustering may be better described in this regime.