Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userThe Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC): Selection of a performance metric for classification probabilities balancing diverse science goals
78
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Classification of transient and variable light curves is an essential step in using astronomical observations to develop an understanding of their underlying physical processes. However, upcoming deep photometric surveys, including the Large Synoptic Survey Telescope (LSST), will produce a deluge of low signal-to-noise data for which traditional labeling procedures are inappropriate. Probabilistic classification is more appropriate for the data but are incompatible with the traditional metrics used on deterministic classifications. Furthermore, large survey collaborations intend to use these classification probabilities for diverse science objectives, indicating a need for a metric that balances a variety of goals. We describe the process used to develop an optimal performance metric for an open classification challenge that seeks probabilistic classifications and must serve many scientific interests. The Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC) is an open competition aiming to identify promising techniques for obtaining classification probabilities of transient and variable objects by engaging a broader community both within and outside astronomy. Using mock classification probability submissions emulating archetypes of those anticipated of PLAsTiCC, we compare the sensitivity of metrics of classification probabilities under various weighting schemes, finding that they yield qualitatively consistent results. We choose as a metric for PLAsTiCC a weighted modification of the cross-entropy because it can be meaningfully interpreted. Finally, we propose extensions of our methodology to ever more complex challenge goals and suggest some guiding principles for approaching the choice of a metric of probabilistic classifications.
rate research
Read More
Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition which aimed to catalyze the development of robust classifiers under LSST-like conditions of a non-representative training set for a large photometric test set of imbalanced classes. Over 1,000 teams participated in PLAsTiCC, which was hosted in the Kaggle data science competition platform between Sep 28, 2018 and Dec 17, 2018, ultimately identifying three winners in February 2019. Participants produced classifiers employing a diverse set of machine learning techniques including hybrid combinations and ensemble averages of a range of approaches, among them boosted decision trees, neural networks, and multi-layer perceptrons. The strong performance of the top three classifiers on Type Ia supernovae and kilonovae represent a major improvement over the current state-of-the-art within astronomy. This paper summarizes the most promising methods and evaluates their results in detail, highlighting future directions both for classifier development and simulation needs for a next generation PLAsTiCC data set.
We describe the simulated data sample for the Photometric LSST Astronomical Time Series Classification Challenge (PLAsTiCC), a publicly available challenge to classify transient and variable events that will be observed by the Large Synoptic Survey Telescope (LSST), a new facility expected to start in the early 2020s. The challenge was hosted by Kaggle, ran from 2018 September 28 to 2018 December 17, and included 1,094 teams competing for prizes. Here we provide details of the 18 transient and variable source models, which were not revealed until after the challenge, and release the model libraries at https://doi.org/10.5281/zenodo.2612896. We describe the LSST Operations Simulator used to predict realistic observing conditions, and we describe the publicly available SNANA simulation code used to transform the models into observed fluxes and uncertainties in the LSST passbands (ugrizy). Although PLAsTiCC has finished, the publicly available models and simulation tools are being used within the astronomy community to further improve classification, and to study contamination in photometrically identified samples of type Ia supernova used to measure properties of dark energy. Our simulation framework will continue serving as a platform to improve the PLAsTiCC models, and to develop new models.
Future surveys such as the Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will observe an order of magnitude more astrophysical transient events than any previous survey before. With this deluge of photometric data, it will be impossible for all such events to be classified by humans alone. Recent efforts have sought to leverage machine learning methods to tackle the challenge of astronomical transient classification, with ever improving success. Transformers are a recently developed deep learning architecture, first proposed for natural language processing, that have shown a great deal of recent success. In this work we develop a new transformer architecture, which uses multi-head self attention at its core, for general multi-variate time-series data. Furthermore, the proposed time-series transformer architecture supports the inclusion of an arbitrary number of additional features, while also offering interpretability. We apply the time-series transformer to the task of photometric classification, minimising the reliance of expert domain knowledge for feature selection, while achieving results comparable to state-of-the-art photometric classification methods. We achieve a weighted logarithmic-loss of 0.507 on imbalanced data in a representative setting using data from the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC). Moreover, we achieve a micro-averaged receiver operating characteristic area under curve of 0.98 and micro-averaged precision-recall area under curve of 0.87.
Space-based transit missions such as Kepler and TESS have demonstrated that planets are ubiquitous. However, the success of these missions heavily depends on ground-based radial velocity (RV) surveys, which combined with transit photometry can yield bulk densities and orbital properties. While most Kepler host stars are too faint for detailed follow-up observations, TESS is detecting planets orbiting nearby bright stars that are more amenable to RV characterization. Here we introduce the TESS-Keck Survey (TKS), an RV program using ~100 nights on Keck/HIRES to study exoplanets identified by TESS. The primary survey aims are investigating the link between stellar properties and the compositions of small planets; studying how the diversity of system architectures depends on dynamical configurations or planet multiplicity; identifying prime candidates for atmospheric studies with JWST; and understanding the role of stellar evolution in shaping planetary systems. We present a fully-automated target selection algorithm, which yielded 103 planets in 86 systems for the final TKS sample. Most TKS hosts are inactive, solar-like, main-sequence stars (4500 K < Teff < 6000 K) at a wide range of metallicities. The selected TKS sample contains 71 small planets (Rp < 4 Re), 11 systems with multiple transiting candidates, 6 sub-day period planets and 3 planets that are in or near the habitable zone of their host star. The target selection described here will facilitate the comparison of measured planet masses, densities, and eccentricities to predictions from planet population models. Our target selection software is publicly available (at https://github.com/ashleychontos/sort-a-survey) and can be adapted for any survey which requires a balance of multiple science interests within a given telescope allocation.
Supernova (SN) classification and redshift estimation using photometric data only have become very important for the Large Synoptic Survey Telescope (LSST), given the large number of SNe that LSST will observe and the impossibility of spectroscopically following up all the SNe. We investigate the performance of a SN classifier that uses SN colors to classify LSST SNe with the Random Forest classification algorithm. Our classifier results in an AUC of 0.98 which represents excellent classification. We are able to obtain a photometric SN sample containing 99$%$ SNe Ia by choosing a probability threshold. We estimate the photometric redshifts (photo-z) of SNe in our sample by fitting the SN light curves using the SALT2 model with nested sampling. We obtain a mean bias ($left<z_mathrm{phot}-z_mathrm{spec}right>$) of 0.012 with $sigmaleft( frac{z_mathrm{phot}-z_mathrm{spec}}{1+z_mathrm{spec}}right) = 0.0294$ without using a host-galaxy photo-z prior, and a mean bias ($left<z_mathrm{phot}-z_mathrm{spec}right>$) of 0.0017 with $sigmaleft( frac{z_mathrm{phot}-z_mathrm{spec}}{1+z_mathrm{spec}}right) = 0.0116$ using a host-galaxy photo-z prior. Assuming a flat $Lambda CDM$ model with $Omega_m=0.3$, we obtain $Omega_m$ of $0.305pm0.008$ (statistical errors only), using the simulated LSST sample of photometric SNe Ia (with intrinsic scatter $sigma_mathrm{int}=0.11$) derived using our methodology without using host-galaxy photo-z prior. Our method will help boost the power of SNe from the LSST as cosmological probes.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
