-
Optimal Compression of Floating-point Astronomical Images Without Significant Loss of Information
Authors:
W. D. Pence,
R. L. White,
R. Seaman
Abstract:
We describe a compression method for floating-point astronomical images that gives compression ratios of 6 -- 10 while still preserving the scientifically important information in the image. The pixel values are first preprocessed by quantizing them into scaled integer intensity levels, which removes some of the uncompressible noise in the image. The integers are then losslessly compressed using t…
▽ More
We describe a compression method for floating-point astronomical images that gives compression ratios of 6 -- 10 while still preserving the scientifically important information in the image. The pixel values are first preprocessed by quantizing them into scaled integer intensity levels, which removes some of the uncompressible noise in the image. The integers are then losslessly compressed using the fast and efficient Rice algorithm and stored in a portable FITS format file. Quantizing an image more coarsely gives greater image compression, but it also increases the noise and degrades the precision of the photometric and astrometric measurements in the quantized image. Dithering the pixel values during the quantization process can greatly improve the precision of measurements in the images. This is especially important if the analysis algorithm relies on the mode or the median which would be similarly quantized if the pixel values are not dithered. We perform a series of experiments on both synthetic and real astronomical CCD images to quantitatively demonstrate that the magnitudes and positions of stars in the quantized images can be measured with the predicted amount of precision. In order to encourage wider use of these image compression methods, we have made available a pair of general-purpose image compression programs, called fpack and funpack, which can be used to compress any FITS format image.
△ Less
Submitted 7 July, 2010;
originally announced July 2010.
-
Optimal DN encoding for CCD detectors
Authors:
Robert L. Seaman,
Richard L. White,
William D. Pence
Abstract:
Image compression has been a frequent topic of presentations at ADASS. Compression is often viewed as just a technique to fit more data into a smaller space. Rather, the packing of data - its "density" - affects every facet of local data handling, long distance data transport, and the end-to-end throughput of workflows. In short, compression is one aspect of proper data structuring. For example,…
▽ More
Image compression has been a frequent topic of presentations at ADASS. Compression is often viewed as just a technique to fit more data into a smaller space. Rather, the packing of data - its "density" - affects every facet of local data handling, long distance data transport, and the end-to-end throughput of workflows. In short, compression is one aspect of proper data structuring. For example, with FITS tile compression the efficient representation of data is combined with an expressive logistical paradigm for its manipulation.
A deeper question remains. Not just how best to represent the data, but which data to represent. CCDs are linear devices. What does this mean? One thing it does not mean is that the analog-to-digital conversion of pixels must be stored using linear data numbers (DN). An alternative strategy of using non- linear representations is presented, with one motivation being to magnify the efficiency of numerical compression algorithms such as Rice.
△ Less
Submitted 19 October, 2009;
originally announced October 2009.
-
Lossless Astronomical Image Compression and the Effects of Noise
Authors:
W. D. Pence,
R. Seaman,
R. L. White
Abstract:
We compare a variety of lossless image compression methods on a large sample of astronomical images and show how the compression ratios and speeds of the algorithms are affected by the amount of noise in the images. In the ideal case where the image pixel values have a random Gaussian distribution, the equivalent number of uncompressible noise bits per pixel is given by Nbits =log2(sigma * sqrt(…
▽ More
We compare a variety of lossless image compression methods on a large sample of astronomical images and show how the compression ratios and speeds of the algorithms are affected by the amount of noise in the images. In the ideal case where the image pixel values have a random Gaussian distribution, the equivalent number of uncompressible noise bits per pixel is given by Nbits =log2(sigma * sqrt(12)) and the lossless compression ratio is given by R = BITPIX / Nbits + K where BITPIX is the bit length of the pixel values and K is a measure of the efficiency of the compression algorithm.
We perform image compression tests on a large sample of integer astronomical CCD images using the GZIP compression program and using a newer FITS tiled-image compression method that currently supports 4 compression algorithms: Rice, Hcompress, PLIO, and GZIP. Overall, the Rice compression algorithm strikes the best balance of compression and computational efficiency; it is 2--3 times faster and produces about 1.4 times greater compression than GZIP. The Rice algorithm produces 75%--90% (depending on the amount of noise in the image) as much compression as an ideal algorithm with K = 0.
The image compression and uncompression utility programs used in this study (called fpack and funpack) are publicly available from the HEASARC web site. A simple command-line interface may be used to compress or uncompress any FITS image file.
△ Less
Submitted 12 March, 2009;
originally announced March 2009.
-
Automated object classification with ClassX
Authors:
A. A. Suchkov,
T. A. McGlynn,
L. Angelini,
M. F. Corcoran,
S. A. Drake,
W. D. Pence,
N. White,
E. L. Winter,
R. J. Hanisch,
R. L. White,
M. Postman,
M. E. Donahue,
F. Genova,
F. Ochsenbein,
P. Fernique,
S. Derriere
Abstract:
ClassX is a project aimed at creating an automated system to classify X-ray sources and is envisaged as a prototype of the Virtual Observatory. As a system, ClassX integrates into a pipeline a network of classifiers and an engine that searches and retrieves for a given target multi-wavelength counterparts from the worldwide data storage media. It applies machine learning methods to `train' diffe…
▽ More
ClassX is a project aimed at creating an automated system to classify X-ray sources and is envisaged as a prototype of the Virtual Observatory. As a system, ClassX integrates into a pipeline a network of classifiers and an engine that searches and retrieves for a given target multi-wavelength counterparts from the worldwide data storage media. It applies machine learning methods to `train' different classifiers using different `training' data sets. In ClassX, each classifier can make its own class (object type) assignment and is optimized for handling different tasks and/or different object types. A user would generally select a certain classifier to make, for instance, a most complete list of candidate QSOs, but a different classifier would be used to make a most reliable list of candidate QSOs. Still different classifiers would be selected to make similar lists for other object types. Along with the class name assignment, a network classifier outputs the probability for a source to belong to the assigned class as well as probabilities that the source belongs in fact to other classes. We illustrate the current capabilities of ClassX and the concept of a classifiers network with the results obtained with classifiers trained using ROSAT data. ~
△ Less
Submitted 17 October, 2002;
originally announced October 2002.
-
Chandra Observation of Luminous and Ultraluminous X-ray Binaries in M101
Authors:
K. Mukai,
W. D. Pence,
S. L. Snowden,
K. D. Kuntz
Abstract:
X-ray binaries in the Milky Way are among the brightest objects on the X-ray sky. With the increasing sensitivity of recent missions, it is now possible to study X-ray binaries in nearby galaxies. We present data on six luminous sources in the nearby spiral galaxy, M101, obtained with the Chandra ACIS-S. Of these, five appear to be similar to ultraluminous sources in other galaxies, while the br…
▽ More
X-ray binaries in the Milky Way are among the brightest objects on the X-ray sky. With the increasing sensitivity of recent missions, it is now possible to study X-ray binaries in nearby galaxies. We present data on six luminous sources in the nearby spiral galaxy, M101, obtained with the Chandra ACIS-S. Of these, five appear to be similar to ultraluminous sources in other galaxies, while the brightest source, P098, shows some unique characteristics. We present our interpretation of the data in terms of an optically thick outflow, and discuss implications.
△ Less
Submitted 9 September, 2002;
originally announced September 2002.
-
Chandra X-ray Sources in M101
Authors:
W. D. Pence,
S. L. Snowden,
K. Mukai,
K. D. Kuntz
Abstract:
A deep (98.2 ks) Chandra Cycle-1 observation has revealed a wealth of discrete X-ray sources as well as diffuse emission in the nearby face-on spiral galaxy M101. From this rich dataset we have created a catalog of the 110 sources from the S3 chip detected with a significance of >3 sigma, corresponding to a flux of ~1.0E-16 ergs/cm/cm/s and a luminosity of 1.0E36 ergs/s for a distance to M101 of…
▽ More
A deep (98.2 ks) Chandra Cycle-1 observation has revealed a wealth of discrete X-ray sources as well as diffuse emission in the nearby face-on spiral galaxy M101. From this rich dataset we have created a catalog of the 110 sources from the S3 chip detected with a significance of >3 sigma, corresponding to a flux of ~1.0E-16 ergs/cm/cm/s and a luminosity of 1.0E36 ergs/s for a distance to M101 of 7.2 Mpc. The sources display a distinct correlation with the spiral arms and include a variety of X-ray binaries, supersoft sources, supernova remnants, and other objects of which only ~27 are likely to be background sources. There are only a few sources in the interarm regions, and most of these have X-ray colors consistent with that of background AGNs. The derived log N-log S relation for the sources in M101 (background subtracted) has a slope of -0.80+/-0.05 over the range of 1.0E36 - 1.0E38 ergs/s. The nucleus is resolved into 2 nearly identical X-ray sources, each with a 0.5-2.0 keV flux of 4.0E37 ergs/s. One of these sources coincides with the optical nucleus, and the other coincides with a cluster of stars 110 pc to the south.
△ Less
Submitted 6 July, 2001;
originally announced July 2001.
