Automated Real-Time Classification and Decision Making in Massive Data Streams from Synoptic Sky Surveys

The nature of scientific and technological data collection is evolving rapidly: data volumes and rates grow exponentially, with increasing complexity and information content, and there has been a transition from static data sets to data streams that must be analyzed in real time. Interesting or anomalous phenomena must be quickly characterized and followed up with additional measurements via optimal deployment of limited assets. Modern astronomy presents a variety of such phenomena in the form of transient events in digital synoptic sky surveys, including cosmic explosions (supernovae, gamma ray bursts), relativistic phenomena (black hole formation, jets), potentially hazardous asteroids, etc. We have been developing a set of machine learning tools to detect, classify and plan a response to transient events for astronomy applications, using the Catalina Real-time Transient Survey (CRTS) as a scientific and methodological testbed. The ability to respond rapidly to the potentially most interesting events is a key bottleneck that limits the scientific returns from the current and anticipated synoptic sky surveys. Similar challenge arise in other contexts, from environmental monitoring using sensor networks to autonomous spacecraft systems. Given the exponential growth of data rates, and the time-critical response, we need a fully automated and robust approach. We describe the results obtained to date, and the possible future developments.

Flashes in a Star Stream: Automated Classification of Astronomical Transient Events

An automated, rapid classification of transient events detected in the modern synoptic sky surveys is essential for their scientific utility and effective follow-up using scarce resources. This presents some unusual challenges: the data are sparse, heterogeneous and incomplete; evolving in time; and most of the relevant information comes not from the data stream itself, but from a variety of archival data and contextual information (spatial, temporal, and multi-wavelength). We are exploring a variety of novel techniques, mostly Bayesian, to respond to these challenges, using the ongoing CRTS sky survey as a testbed. The current surveys are already overwhelming our ability to effectively follow all of the potentially interesting events, and these challenges will grow by orders of magnitude over the next decade as the more ambitious sky surveys get under way. While we focus on an application in a specific domain (astrophysics), these challenges are more broadly relevant for event or anomaly detection and knowledge discovery in massive data streams.

The Discovery and Nature of Optical Transient CSS100217:102913+404220

We report on the discovery and observations of the extremely luminous optical transient CSS100217:102913+404220 (CSS100217 hereafter). Spectroscopic observations show this transient was coincident with a galaxy at redshift z=0.147, and reached an apparent magnitude of V ~ 16.3. After correcting for foreground Galactic extinction we determine the absolute magnitude to be M_V =-22.7 approximately 45 days after maximum light. Based on our unfiltered optical photometry the peak optical emission was L = 1.3 x 10^45 erg s^-1, and over a period of 287 rest-frame days had an integrated bolometric luminosity of 1.2 x 10^52 erg. Analysis of the pre-outburst SDSS spectrum of the source shows features consistent with a Narrow-line Seyfert1 (NLS1) galaxy. High-resolution HST and Keck followup observations show the event occurred within 150pc of nucleus of the galaxy, suggesting a possible link to the active nuclear region. However, the rapid outburst along with photometric and spectroscopic evolution are much more consistent with a luminous supernova. Line diagnostics suggest that the host galaxy is undergoing significant star formation. We use extensive follow-up of the event along with archival CSS and SDSS data to investigate the three most likely sources of such an event; 1) an extremely luminous supernova; 2) the tidal disruption of a star by the massive nuclear black hole; 3) variability of the central AGN. We find that CSS100217 was likely an extremely luminous type IIn supernova that occurred within range of the narrow-line region of an AGN. We discuss how similar events may have been missed in past supernova surveys because of confusion with AGN activity.

First Results from the Catalina Real-time Transient Survey

We report on the results from the first six months of the Catalina Real-time Transient Survey (CRTS). In order to search for optical transients with timescales of minutes to years, the CRTS analyses data from the Catalina Sky Survey which repeatedly covers twenty six thousand of square degrees on the sky. The CRTS provides a public stream of transients that are bright enough to be followed up using small telescopes. Since the beginning of the survey, all CRTS transients have been made available to astronomers around the world in real-time using HTML tables, RSS feeds and VOEvents. As part of our public outreach program the detections are now also available in KML through Google Sky. The initial discoveries include over 350 unique optical transients rising more than two magnitudes from past measurements. Sixty two of these are classified as supernovae, based on light curves, prior deep imaging and spectroscopic data. Seventy seven are due to cataclysmic variables (only 13 previously known), while an additional 100 transients were too infrequently sampled to distinguish between faint CVs and SNe. The remaining optical transients include AGN, Blazars, high proper motions stars, highly variable stars (such as UV Ceti stars) and transients of an unknown nature. Our results suggest that there is a large population of SNe missed by many current supernova surveys because of selection biases. These objects appear to be associated with faint host galaxies. We also discuss the unexpected discovery of white dwarf binary systems through dramatic eclipses.