Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAutomated Real-Time Classification and Decision Making in Massive Data Streams from Synoptic Sky Surveys
154
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
The nature of scientific and technological data collection is evolving rapidly: data volumes and rates grow exponentially, with increasing complexity and information content, and there has been a transition from static data sets to data streams that must be analyzed in real time. Interesting or anomalous phenomena must be quickly characterized and followed up with additional measurements via optimal deployment of limited assets. Modern astronomy presents a variety of such phenomena in the form of transient events in digital synoptic sky surveys, including cosmic explosions (supernovae, gamma ray bursts), relativistic phenomena (black hole formation, jets), potentially hazardous asteroids, etc. We have been developing a set of machine learning tools to detect, classify and plan a response to transient events for astronomy applications, using the Catalina Real-time Transient Survey (CRTS) as a scientific and methodological testbed. The ability to respond rapidly to the potentially most interesting events is a key bottleneck that limits the scientific returns from the current and anticipated synoptic sky surveys. Similar challenge arise in other contexts, from environmental monitoring using sensor networks to autonomous spacecraft systems. Given the exponential growth of data rates, and the time-critical response, we need a fully automated and robust approach. We describe the results obtained to date, and the possible future developments.
rate research
Read More
The nature of scientific and technological data collection is evolving rapidly: data volumes and rates grow exponentially, with increasing complexity and information content, and there has been a transition from static data sets to data streams that must be analyzed in real time. Interesting or anomalous phenomena must be quickly characterized and followed up with additional measurements via optimal deployment of limited assets. Modern astronomy presents a variety of such phenomena in the form of transient events in digital synoptic sky surveys, including cosmic explosions (supernovae, gamma ray bursts), relativistic phenomena (black hole formation, jets), potentially hazardous asteroids, etc. We have been developing a set of machine learning tools to detect, classify and plan a response to transient events for astronomy applications, using the Catalina Real-time Transient Survey (CRTS) as a scientific and methodological testbed. The ability to respond rapidly to the potentially most interesting events is a key bottleneck that limits the scientific returns from the current and anticipated synoptic sky surveys. Similar challenge arise in other contexts, from environmental monitoring using sensor networks to autonomous spacecraft systems. Given the exponential growth of data rates, and the time-critical response, we need a fully automated and robust approach. We describe the results obtained to date, and the possible future developments.
We compare the performance of multiple codes written by different groups for making polarized maps from Planck-sized, all-sky cosmic microwave background (CMB) data. Three of the codes are based on a destriping algorithm; the other three are implementations of an optimal maximum-likelihood algorithm. Time-ordered data (TOD) were simulated using the Planck Level-S simulation pipeline. Several cases of temperature-only data were run to test that the codes could handle large datasets, and to explore effects such as the precision of the pointing data. Based on these preliminary results, TOD were generated for a set of four 217 GHz detectors (the minimum number required to produce I, Q, and U maps) under two different scanning strategies, with and without noise. Following correction of various problems revealed by the early simulation, all codes were able to handle the large data volume that Planck will produce. Differences in maps produced are small but noticeable; differences in computing resources are large.
We are proposing to conduct a multicolor, synoptic infrared (IR) imaging survey of the Northern sky with a new, dedicated 6.5-meter telescope at San Pedro Martir (SPM) Observatory. This initiative is being developed in partnership with astronomy institutions in Mexico and the University of California. The 4-year, dedicated survey, planned to begin in 2017, will reach more than 100 times deeper than 2MASS. The Synoptic All-Sky Infrared (SASIR) Survey will reveal the missing sample of faint red dwarf stars in the local solar neighborhood, and the unprecedented sensitivity over such a wide field will result in the discovery of thousands of z ~ 7 quasars (and reaching to z > 10), allowing detailed study (in concert with JWST and Giant Segmented Mirror Telescopes) of the timing and the origin(s) of reionization. As a time-domain survey, SASIR will reveal the dynamic infrared universe, opening new phase space for discovery. Synoptic observations of over 10^6 supernovae and variable stars will provide better distance measures than optical studies alone. SASIR also provides significant synergy with other major Astro2010 facilities, improving the overall scientific return of community investments. Compared to optical-only measurements, IR colors vastly improve photometric redshifts to z ~ 4, enhancing dark energy and dark matter surveys based on weak lensing and baryon oscillations. The wide field and ToO capabilities will enable a connection of the gravitational wave and neutrino universe - with events otherwise poorly localized on the sky - to transient electromagnetic phenomena.
The direct detection of binary systems in wide-field surveys is limited by the size of the stars point-spread-functions (PSFs). A search for elongated objects can find closer companions, but is limited by the precision to which the PSF shape can be calibrated for individual stars. We have developed the BinaryFinder algorithm to search for close binaries by using precision measurements of PSF ellipticity across wide-field survey images. We show that the algorithm is capable of reliably detecting binary systems down to approximately 1/5 of the seeing limit, and can directly measure the systems position angles, separations and contrast ratios. To verify the algorithms performance we evaluated 100,000 objects in Palomar Transient Factory (PTF) wide-field-survey data for signs of binarity, and then used the Robo-AO robotic laser adaptive optics system to verify the parameters of 44 high-confidence targets. We show that BinaryFinder correctly predicts the presence of close companions with a <5% false-positive rate, measures the detected binaries position angles within 2 degrees and separations within 25%, and weakly constrains their contrast ratios. When applied to the full PTF dataset, we estimate that BinaryFinder will discover and characterize ~450,000 physically-associated binary systems with separations <2 arcseconds and magnitudes brighter than R=18. New wide-field synoptic surveys with high sensitivity and sub-arcsecond angular resolution, such as LSST, will allow BinaryFinder to reliably detect millions of very faint binary systems with separations as small as 0.1 arcseconds.
Event-based cameras are novel, efficient sensors inspired by the human vision system, generating an asynchronous, pixel-wise stream of data. Learning from such data is generally performed through heavy preprocessing and event integration into images. This requires buffering of possibly long sequences and can limit the response time of the inference system. In this work, we instead propose to directly use events from a DVS camera, a stream of intensity changes and their spatial coordinates. This sequence is used as the input for a novel emph{asynchronous} RNN-like architecture, the Input-filtering Neural ODEs (INODE). This is inspired by the dynamical systems and filtering literature. INODE is an extension of Neural ODEs (NODE) that allows for input signals to be continuously fed to the network, like in filtering. The approach naturally handles batches of time series with irregular time-stamps by implementing a batch forward Euler solver. INODE is trained like a standard RNN, it learns to discriminate short event sequences and to perform event-by-event online inference. We demonstrate our approach on a series of classification tasks, comparing against a set of LSTM baselines. We show that, independently of the camera resolution, INODE can outperform the baselines by a large margin on the ASL task and its on par with a much larger LSTM for the NCALTECH task. Finally, we show that INODE is accurate even when provided with very few events.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
