There is currently a severe discrepancy between theoretical models of dust formation in core-collapse supernovae (CCSNe), which predict $gtrsim 0.01$ M$_odot$ of ejecta dust forming within $sim 1000$ days, and observations at these epochs, which infer much lower masses. We demonstrate that, in the optically thin case, these low dust masses are robust despite significant observational and model uncertainties. For a sample of 11 well-observed CCSNe, no plausible model reaches carbon dust masses above $10^{-4}$ M$_odot$, or silicate masses above $sim 10^{-3}$ M$_odot$. Optically thick models can accommodate larger dust masses, but the dust must be clumped and have a low ($<0.1$) covering fraction to avoid conflict with data at optical wavelengths. These values are insufficient to reproduce the observed infrared fluxes, and the required covering fraction varies not only between SNe but between epochs for the same object. The difficulty in reconciling large dust masses with early-time observations of CCSNe, combined with well-established detections of comparably large dust masses in supernova remnants, suggests that a mechanism for late-time dust formation is necessary.