Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA Redistribution Tool for Long-Term Archive of Astronomical Observation Data
113
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Astronomical observation data require long-term preservation, and the rapid accumulation of observation data makes it necessary to consider the cost of long-term archive storage. In addition to low-speed disk-based online storage, optical disk or tape-based offline storage can be used to save costs. However, for astronomical research that requires historical data (particularly time-domain astronomy), the performance and energy consumption of data-accessing techniques cause problems because the requested data (which are organized according to observation time) may be located across multiple storage devices. In this study, we design and develop a tool referred to as AstroLayout to redistribute the observation data using spatial aggregation. The core algorithm uses graph partitioning to generate an optimized data placement according to the original observation data statistics and the target storage system. For the given observation data, AstroLayout can copy the long-term archive in the target storage system in accordance with this placement. An efficiency evaluation shows that AstroLayout can reduce the number of devices activated when responding to data-access requests in time-domain astronomy research. In addition to improving the performance of data-accessing techniques, AstroLayout can also reduce the storage systems power consumption. For enhanced adaptability, it supports storage systems of any media, including optical disks, tapes, and hard disks.
rate research
Read More
Modern surveys have provided the astronomical community with a flood of high-dimensional data, but analyses of these data often occur after their projection to lower-dimensional spaces. In this work, we introduce a local two-sample hypothesis test framework that an analyst may directly apply to data in their native space. In this framework, the analyst defines two classes based on a response variable of interest (e.g. higher-mass galaxies versus lower-mass galaxies) and determines at arbitrary points in predictor space whether the local proportions of objects that belong to the two classes significantly differs from the global proportion. Our framework has a potential myriad of uses throughout astronomy; here, we demonstrate its efficacy by applying it to a sample of 2487 i-band-selected galaxies observed by the HST ACS in four of the CANDELS program fields. For each galaxy, we have seven morphological summary statistics along with an estimated stellar mass and star-formation rate. We perform two studies: one in which we determine regions of the seven-dimensional space of morphological statistics where high-mass galaxies are significantly more numerous than low-mass galaxies, and vice-versa, and another study where we use SFR in place of mass. We find that we are able to identify such regions, and show how high-mass/low-SFR regions are associated with concentrated and undisturbed galaxies while galaxies in low-mass/high-SFR regions appear more extended and/or disturbed than their high-mass/low-SFR counterparts.
A concept of the ground-based optical astronomical observations efficiency is considered in this paper. We believe that a telescope efficiency can be increased by properly allocating observation tasks with respect to the current environment state and probability to obtain the data with required properties under the current conditions. An online observations scheduling is assumed to be essential part for raising the efficiency. The short-term online scheduling is treated as the discrete optimisation problems which are stated using several abstraction levels. The optimisation problems are solved using a parallel depth-bounded discrepancy search (PDDS) algorithm [13]. Some aspects of the algorithm performance are discussed. The presented algorithm is a core of open-source chelyabinsk C++ library which is supposed to be used at 2.5 m telescope of Sternberg Astronomical Institude of Lomonosov Moscow State University.
Dome A, Antarctica has been thought to be one of the best astronomical sites on the earth since decades ago. From it was first visited by astronomers in 2008, dozens of facilities for astronomical observation and site testing were deployed. Due to its special geographical location, the data and message exchange between Dome A and the domestic control center could only depend on Iridium. Because the link bandwidth of Iridium is extremely limited, meanwhile the network traffic cost is quite expensive and the network is rather unstable, the commonly used data transfer tools, such as rsync and scp, are not suitable in this case. In this paper, we design and implement a data transfer tool called NBFTP (narrow bandwidth file transfer protocol) for the astronomical observation of Dome A. NBFTP uses a uniform interface to arrange all types of data and matches specific transmission schemes for different data types according to rules. Break-point resuming and extensibility functions are also implemented. Our experimental results show that NBFTP consumes 60% less network traffic than rsync when detecting the data pending to be transferred. And when transferring small files of 1KB, the network traffic consumption of NBFTP is 40% less than rsync. However, as the file size increases, the network traffic consumption of NBFTP tends to approach rsync, but it is still smaller than rsync.
NASA regards data handling and archiving as an integral part of space missions, and has a strong track record of serving astrophysics data to the public, beginning with the the IRAS satellite in 1983. Archives enable a major science return on the significant investment required to develop a space mission. In fact, the presence and accessibility of an archive can more than double the number of papers resulting from the data. In order for the community to be able to use the data, they have to be able to find the data (ease of access) and interpret the data (ease of use). Funding of archival research (e.g., the ADAP program) is also important not only for making scientific progress, but also for encouraging authors to deliver data products back to the archives to be used in future studies. NASA has also enabled a robust system that can be maintained over the long term, through technical innovation and careful attention to resource allocation. This article provides a brief overview of some of NASAs major astrophysics archive systems, including IRSA, MAST, HEASARC, KOA, NED, the Exoplanet Archive, and ADS.
From the moment astronomical observations are made the resulting data products begin to grow stale. Even if perfect binary copies are preserved through repeated timely migration to more robust storage media, data standards evolve and new tools are created that require different kinds of data or metadata. The expectations of the astronomical community change even if the data do not. We discuss data engineering to mitigate the ensuing risks with examples from a recent project to refactor seven million archival images to new standards of nomenclature, metadata, format, and compression.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
