Article ID | Journal | Published Year | Pages | File Type |
---|---|---|---|---|
8141255 | New Astronomy | 2018 | 8 Pages |
Abstract
Astronomical instruments commonly generate large data series in tabular format. To efficiently store and transmit these data series, carefully designed compression formats are welcome. We propose DeltaComp, a free open-source program suited for (relatively) smooth streams of data. DeltaComp is based on rather simple mechanisms; essentially it is a tailored combination of delta coding and context-based modeling. Following the methodology of the preceding work presenting Polycomp (Tomasi, 2016), we compress (i) the ephemeris table of Ganymede, and (ii) the publicly available timelines recorded by LFI, an array of microwave radiometers aboard the ESA Planck spacecraft. In the former case (with small data) the compression ratio advantage of DeltaComp over Polycomp is by a factor of about 1.4. For the Planck data, of size 4.24TB, the archives produced by DeltaComp are almost six times smaller than those from Polycomp, at somewhat smaller median quantization error (and much smaller maximum error), which translates to only 66GB of required storage. Importantly, DeltaComp is over three orders of magnitude faster than Polycomp.
Related Topics
Physical Sciences and Engineering
Physics and Astronomy
Astronomy and Astrophysics
Authors
Sebastian Deorowicz, Szymon Grabowski,