Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userUsing Binary File Format Description Languages for Documenting, Parsing, and Verifying Raw Data in TAIGA Experiment
213
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
The paper is devoted to the issues of raw binary data documenting, parsing and verifying in astroparticle data lifecycle. The long-term preservation of raw data of astroparticle experiments as originally generated is essential for re-running analyses and reproducing research results. The selected high-quality raw data should have detailed documentation and accompanied by open software tools for access to them. We consider applicability of binary file format description languages to specify, parse and verify raw data of the Tunka Advanced Instrument for cosmic rays and Gamma Astronomy (TAIGA) experiment. The formal specifications are implemented for five data formats of the experiment and provide automatic generation of source code for data reading libraries in target programming languages (e.g. C++, Java, and Python). These libraries were tested on TAIGA data. They showed a good performance and help us to locate the parts with corrupted data. The format specifications can be used as metadata for exchanging of astroparticle raw data. They can also simplify software development for data aggregation from various sources for the multi-messenger analysis.
rate research
Read More
In order to allow different software applications, in constant evolution, to interact and exchange data, flexible file formats are needed. A file format specification for different types of content has been elaborated to allow communication of data for the software developed within the European Network of Excellence NANOQUANTA, focusing on first-principles calculations of materials and nanosystems. It might be used by other software as well, and is described here in detail. The format relies on the NetCDF binary input/output library, already used in many different scientific communities, that provides flexibility as well as portability accross languages and platforms. Thanks to NetCDF, the content can be accessed by keywords, ensuring the file format is extensible and backward compatible.
NeXus is an international standard data format intended to reduce the need for redundant software development efforts in the neutron and x-ray scattering communities. As the NeXus standard matures it is starting to be used at laboratories for storing raw data. The Manuel Lujan Jr. Neutron Scattering Center (MLNSC) at Los Alamos National Laboratory and the Intense Pulsed Neutron Source (IPNS) at Argonne National Laboratory have been working with NeXus in an effort to share data and software. MLNSC is now writing files compliant with NeXus and the Integrated Spectral Analysis Workbench (ISAW) software from IPNS is being used with this data. Problems can arise if the standard is interpreted in different ways and information that belongs in the file is not accounted for in the standard. This paper will discuss an inter-laboratory collaboration in relation to a maturing data standard.
The IAU Commission 4 Working Group on Standardizing Access to Ephemerides recommends the use of the Spacecraft and Planet Kernel (SPK) format as a standard format for the position ephemerides of planets and other natural solar system bodies, and the use of the Planetary Constants Kernel (PCK) format for the orientation of these bodies. It further recommends that other supporting data be stored in a text PCK. These formats were developed for use by the SPICE Toolkit by the Navigation and Ancillary Information Facility of NASAs Jet Propulsion Laboratory (JPL). The CALCEPH library developed by the Institut de mecanique celeste de calcul des ephemerides (IMCCE) is also able to make use of these files. High accuracy ephemerides available in files conforming to the SPK and PCK formats include: the Development Ephemerides (DE) from JPL, Integrateur Numerique Planetaire de lObservatoire de Paris (INPOP) from IMCCE, and the Ephemerides Planets and the Moon (EPM), developed by the Institute for Applied Astronomy (IAA). The bulk of this report is a description of the portion of PCK and SPK formats required for these ephemerides. New SPK and PCK data types, both called Type 20: Chebyshev (Velocity Only), have been added. Other changes to the specification are (i) a new object identification number for coordinate time ephemerides and (ii) a set of three new data types that use the TCB rather than the TDB time scale for the ephemerides, but are otherwise identical to their T
The General Single-Dish Data format (GSDD) was developed in the mid-1980s as a data model to support centimeter, millimeter and submillimeter instrumentation at NRAO, JCMT, the University of Arizona and IRAM. We provide an overview of the GSDD requirements and associated data model, discuss the implementation of the resultant file formats, describe its usage in the observatories and provide a retrospective on the format.
Nowadays astroparticle physics faces a rapid data volume increase. Meanwhile, there are still challenges of testing the theoretical models for clarifying the origin of cosmic rays by applying a multi-messenger approach, machine learning and investigation of the phenomena related to the rare statistics in detecting incoming particles. The problems are related to the accurate data mapping and data management as well as to the distributed storage and high-performance data processing. In particular, one could be interested in employing such solutions in study of air-showers induced by ultra-high energy cosmic and gamma rays, testing new hypotheses of hadronic interaction or cross-calibration of different experiments. KASCADE (Karlsruhe, Germany) and TAIGA (Tunka valley, Russia) are experiments in the field of astroparticle physics, aiming at the detection of cosmic-ray air-showers, induced by the primaries in the energy range of about hundreds TeVs to hundreds PeVs. They are located at the same latitude and have an overlap in operation runs. These factors determine the interest in performing a joint analysis of these data. In the German-Russian Astroparticle Data Life Cycle Initiative (GRADLCI), modern technologies of the distributed data management are being employed for establishing a reliable open access to the experimental cosmic-ray physics data collected by KASCADE and the Tunka-133 setup of TAIGA.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
