We present a procedure for efficiently compressing astronomical radio data for high performance applications. Integrated, post-correlation data are first passed through a nearly lossless rounding step which compares the precision of the data to a gen
eralized and calibration-independent form of the radiometer equation. This allows the precision of the data to be reduced in a way that has an insignificant impact on the data. The newly developed Bitshuffle lossless compression algorithm is subsequently applied. When the algorithm is used in conjunction with the HDF5 library and data format, data produced by the CHIME Pathfinder telescope is compressed to 28% of its original size and decompression throughputs in excess of 1 GB/s are obtained on a single core.
In this letter, 21 cm intensity maps acquired at the Green Bank Telescope are cross-correlated with large-scale structure traced by galaxies in the WiggleZ Dark Energy Survey. The data span the redshift range 0.6 < z < 1 over two fields totaling ~41
deg. sq. and 190 hours of radio integration time. The cross-correlation constrains Omega_HI b_HI r = [0.43 pm 0.07 (stat.) pm 0.04(sys.)] x 10^-3, where Omega_HI is the neutral hydrogen HI fraction, r is the galaxy-hydrogen correlation coefficient, and b_HI is the HI bias parameter. This is the most precise constraint on neutral hydrogen density fluctuations in a challenging redshift range. Our measurement improves the previous 21 cm cross-correlation at z ~ 0.8 both in its precision and in the range of scales probed.
