Anomaly detection for machine learning redshifts applied to SDSS galaxies

We present an analysis of anomaly detection for machine learning redshift estimation. Anomaly detection allows the removal of poor training examples, which can adversely influence redshift estimates. Anomalous training examples may be photometric galaxies with incorrect spectroscopic redshifts, or galaxies with one or more poorly measured photometric quantity. We select 2.5 million clean SDSS DR12 galaxies with reliable spectroscopic redshifts, and 6730 anomalous galaxies with spectroscopic redshift measurements which are flagged as unreliable. We contaminate the clean base galaxy sample with galaxies with unreliable redshifts and attempt to recover the contaminating galaxies using the Elliptical Envelope technique. We then train four machine learning architectures for redshift analysis on both the contaminated sample and on the preprocessed anomaly-removed sample and measure redshift statistics on a clean validation sample generated without any preprocessing. We find an improvement on all measured statistics of up to 80% when training on the anomaly removed sample as compared with training on the contaminated sample for each of the machine learning routines explored. We further describe a method to estimate the contamination fraction of a base data sample.

The Self-Calibrating Hubble Diagram

As an increasing number of well measured type Ia supernovae (SNe Ia) become available, the statistical uncertainty on w has been reduced to the same size as the systematic uncertainty. The statistical error will decrease further in the near future, and hence the improvement of systematic uncertainties needs to be addressed, if further progress is to be made. We study how uncertainties in the primary reference spectrum - which are a main contribution to the systematic uncertainty budget - affect the measurement of the Dark Energy equation of state parameter w from SNe Ia. The increasing number of SN observations can be used to reduce the uncertainties by including perturbations of the reference spectrum as nuisance parameters in a cosmology fit, thus self-calibrating the Hubble diagram. We employ this method to real SNe data for the first time and find the perturbations of the reference spectrum consistent with zero at the 1%-level. For future surveys we estimate that ~3500 SNe will be required for our method to outperform the standard method of deriving the cosmological parameters.