Accurate inference of cosmology from weak lensing shear requires an accurate shear power spectrum covariance matrix. Here, we investigate this accuracy requirement and quantify the relative importance of the Gaussian (G), super-sample covariance (SSC) and connected non-Gaussian (cNG) contributions to the covariance. Specifically, we forecast cosmological parameter constraints for future wide-field surveys and study how different covariance matrix components affect parameter bounds. Our main result is that the cNG term represents only a small and potentially negligible contribution to statistical parameter errors: the errors obtained using the G+SSC subset are within $lesssim 5%$ of those obtained with the full G+SSC+cNG matrix for a Euclid-like survey. This result also holds for the shear two-point correlation function, variations in survey specifications and for different analytical prescriptions of the cNG term. The cNG term is that which is often tackled using numerically expensive ensembles of survey realizations. Our results suggest however that the accuracy of analytical or approximate numerical methods to compute the cNG term is likely to be sufficient for cosmic shear inference from the next generation of surveys.