Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAn approach to the analysis of SDSS spectroscopic outliers based on Self-Organizing Maps
106
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Aims. A new method is applied to the segmentation, and further analysis of the outliers resulting from the classification of astronomical objects in large databases is discussed. The method is being used in the framework of the Gaia satellite DPAC (Data Processing and Analysis Consortium) activities to prepare automated software tools that will be used to derive basic astrophysical information that is to be included in Gaia final archive. Methods. Our algorithm has been tested by means of simulated Gaia spectrophotometry, which is based on SDSS observations and theoretical spectral libraries covering a wide sample of astronomical objects. Self-Organizing Maps (SOM) networks are used to organize the information in clusters of objects, as homogeneous as possible, according to their spectral energy distributions (SED), and to project them onto a 2-D grid where the data structure can be visualized. Results. We demonstrate the usefulness of the method by analyzing the spectra that were rejected by the SDSS spectroscopic classification pipeline and thus classified as UNKNOWN. Firstly, our method can help to distinguish between astrophysical objects and instrumental artifacts. Additionally, the application of our algorithm to SDSS objects of unknown nature has allowed us to identify classes of objects of similar astrophysical nature. In addition, the method allows for the potential discovery of hundreds of novel objects, such as white dwarfs and quasars. Therefore, the proposed method is shown to be very promising for data exploration and knowledge discovery in very large astronomical databases, such as the upcoming Gaia mission.
rate research
Read More
Self-Organizing Map (SOM) is a promising tool for exploring large multi-dimensional data sets. It is quick and convenient to train in an unsupervised fashion and, as an outcome, it produces natural clusters of data patterns. An example of application of SOM to the new OGLE-III data set is presented along with some preliminary results. Once tested on OGLE data, the SOM technique will also be implemented within the Gaia missions photometry and spectrometry analysis, in particular, in so-called classification-based Science Alerts. SOM will be used as a basis of this system as the changes in brightness and spectral behaviour of a star can be easily and quickly traced on a map trained in advance with simulated and/or real data from other surveys.
Generating interpretable visualizations from complex data is a common problem in many applications. Two key ingredients for tackling this issue are clustering and representation learning. However, current methods do not yet successfully combine the strengths of these two approaches. Existing representation learning models which rely on latent topological structure such as self-organising maps, exhibit markedly lower clustering performance compared to recent deep clustering methods. To close this performance gap, we (a) present a novel way to fit self-organizing maps with probabilistic cluster assignments (PSOM), (b) propose a new deep architecture for probabilistic clustering (DPSOM) using a VAE, and (c) extend our architecture for time-series clustering (T-DPSOM), which also allows forecasting in the latent space using LSTMs. We show that DPSOM achieves superior clustering performance compared to current deep clustering methods on MNIST/Fashion-MNIST, while maintaining the favourable visualization properties of SOMs. On medical time series, we show that T-DPSOM outperforms baseline methods in time series clustering and time series forecasting, while providing interpretable visualizations of patient state trajectories and uncertainty estimation.
We discuss a diffusion based implementation of the self-organizing map on the unit hypersphere. We show that this approach can be efficiently implemented using just linear algebra methods, we give a python numpy implementation, and we illustrate the approach using the well known MNIST dataset.
Wide-field imaging surveys such as the Dark Energy Survey (DES) rely on coarse measurements of spectral energy distributions in a few filters to estimate the redshift distribution of source galaxies. In this regime, sample variance, shot noise, and selection effects limit the attainable accuracy of redshift calibration and thus of cosmological constraints. We present a new method to combine wide-field, few-filter measurements with catalogs from deep fields with additional filters and sufficiently low photometric noise to break degeneracies in photometric redshifts. The multi-band deep field is used as an intermediary between wide-field observations and accurate redshifts, greatly reducing sample variance, shot noise, and selection effects. Our implementation of the method uses self-organizing maps to group galaxies into phenotypes based on their observed fluxes, and is tested using a mock DES catalog created from N-body simulations. It yields a typical uncertainty on the mean redshift in each of five tomographic bins for an idealized simulation of the DES Year 3 weak-lensing tomographic analysis of $sigma_{Delta z} = 0.007$, which is a 60% improvement compared to the Year 1 analysis. Although the implementation of the method is tailored to DES, its formalism can be applied to other large photometric surveys with a similar observing strategy.
Unsupervised machine learning is one of the main techniques employed in artificial intelligence. Quantum computers offer opportunities to speed up such machine learning techniques. Here, we introduce an algorithm for quantum assisted unsupervised data clustering using the self-organizing feature map, a type of artificial neural network. We make a proof-of-concept realization of one of the central components on the IBM Q Experience and show that it allows us to reduce the number of calculations in a number of clusters. We compare the results with the classical algorithm on a toy example of unsupervised text clustering.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
