The structure and design of optimal supply networks is an important topic in complex networks research. A fundamental trait of natural and man-made networks is the emergence of loops and the trade-off governing their formation: adding redundant edges to supply networks is costly, yet beneficial for resilience. Loops typically form when costs for new edges are small or inputs uncertain. Here, we shed further light on the transition to loop formation. We demonstrate that loops emerge discontinuously when decreasing the costs for new edges for both an edge-damage model and a fluctuating sink model. Mathematically, new loops are shown to form through a saddle-node bifurcation. Our analysis allows to heuristically predict the location and cost where the first loop emerges. Finally, we unveil an intimate relationship among betweenness measures and optimal tree networks. Our results can be used to understand the evolution of loop formation in real-world biological networks.