The competition between superconductivity and charge density wave (CDW) order in underdoped cuprates has now been widely reported, but the role of disorder in this competition has yet to be fully resolved. A central question is whether disorder sets the length scale of the CDW order, for instance by pinning charge density fluctuations or disrupting an otherwise long range order. Using resonant soft x-ray scattering, we investigate the sensitivity of CDW order in YBa$_2$Cu$_3$O$_{6+x}$ (YBCO) to varying levels of oxygen disorder. We find that quench cooling YBCO$_{6.67}$ (YBCO$_{6.75}$) crystals to destroy their o-V and o-VIII (o-III) chains decreases the intensity of the CDW superlattice peak by a factor of 1.9 (1.3), but has little effect on the CDW correlation length, incommensurability, and temperature dependence. This reveals that while quenched oxygen disorder influences the CDW order parameter, the spatial extent of the CDW order is insensitive to the level of quenched oxygen disorder and may instead be a consequence of competition with superconductivity.