The $sigma$ Orionis cluster is important for studying protoplanetary disk evolution, as its intermediate age ($sim$3-5 Myr) is comparable to the median disk lifetime. We use ALMA to conduct a high-sensitivity survey of dust and gas in 92 protoplanetary disks around $sigma$ Orionis members with $M_{ast}gtrsim0.1 M_{odot}$. Our observations cover the 1.33 mm continuum and several CO $J=2-1$ lines: out of 92 sources, we detect 37 in the mm continuum and six in $^{12}$CO, three in $^{13}$CO, and none in C$^{18}$O. Using the continuum emission to estimate dust mass, we find only 11 disks with $M_{rm dust}gtrsim10 M_{oplus}$, indicating that after only a few Myr of evolution most disks lack sufficient dust to form giant planet cores. Stacking the individually undetected continuum sources limits their average dust mass to 5$times$ lower than that of the faintest detected disk, supporting theoretical models that indicate rapid dissipation once disk clearing begins. Comparing the protoplanetary disk population in $sigma$ Orionis to those of other star-forming regions supports the steady decline in average dust mass and the steepening of the $M_{rm dust}$-$M_{ast}$ relation with age; studying these evolutionary trends can inform the relative importance of different disk processes during key eras of planet formation. External photoevaporation from the central O9 star is influencing disk evolution throughout the region: dust masses clearly decline with decreasing separation from the photoionizing source, and the handful of CO detections exist at projected separations $>1.5$ pc. Collectively, our findings indicate that giant planet formation is inherently rare and/or well underway by a few Myr of age.