-
The LOFAR Two-metre Sky Survey -- V. Second data release
Authors:
T. W. Shimwell,
M. J. Hardcastle,
C. Tasse,
P. N. Best,
H. J. A. Röttgering,
W. L. Williams,
A. Botteon,
A. Drabent,
A. Mechev,
A. Shulevski,
R. J. van Weeren,
L. Bester,
M. Brüggen,
G. Brunetti,
J. R. Callingham,
K. T. Chyży,
J. E. Conway,
T. J. Dijkema,
K. Duncan,
F. de Gasperin,
C. L. Hale,
M. Haverkorn,
B. Hugo,
N. Jackson,
M. Mevius
, et al. (81 additional authors not shown)
Abstract:
In this data release from the LOFAR Two-metre Sky Survey (LoTSS) we present 120-168MHz images covering 27% of the northern sky. Our coverage is split into two regions centred at approximately 12h45m +44$^\circ$30' and 1h00m +28$^\circ$00' and spanning 4178 and 1457 square degrees respectively. The images were derived from 3,451hrs (7.6PB) of LOFAR High Band Antenna data which were corrected for th…
▽ More
In this data release from the LOFAR Two-metre Sky Survey (LoTSS) we present 120-168MHz images covering 27% of the northern sky. Our coverage is split into two regions centred at approximately 12h45m +44$^\circ$30' and 1h00m +28$^\circ$00' and spanning 4178 and 1457 square degrees respectively. The images were derived from 3,451hrs (7.6PB) of LOFAR High Band Antenna data which were corrected for the direction-independent instrumental properties as well as direction-dependent ionospheric distortions during extensive, but fully automated, data processing. A catalogue of 4,396,228 radio sources is derived from our total intensity (Stokes I) maps, where the majority of these have never been detected at radio wavelengths before. At 6" resolution, our full bandwidth Stokes I continuum maps with a central frequency of 144MHz have: a median rms sensitivity of 83$μ$Jy/beam; a flux density scale accuracy of approximately 10%; an astrometric accuracy of 0.2"; and we estimate the point-source completeness to be 90% at a peak brightness of 0.8mJy/beam. By creating three 16MHz bandwidth images across the band we are able to measure the in-band spectral index of many sources, albeit with an error on the derived spectral index of +/-0.2 which is a consequence of our flux-density scale accuracy and small fractional bandwidth. Our circular polarisation (Stokes V) 20" resolution 120-168MHz continuum images have a median rms sensitivity of 95$μ$Jy/beam, and we estimate a Stokes I to Stokes V leakage of 0.056%. Our linear polarisation (Stokes Q and Stokes U) image cubes consist of 480 x 97.6 kHz wide planes and have a median rms sensitivity per plane of 10.8mJy/beam at 4' and 2.2mJy/beam at 20"; we estimate the Stokes I to Stokes Q/U leakage to be approximately 0.2%. Here we characterise and publicly release our Stokes I, Q, U and V images in addition to the calibrated uv-data.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Realising the LOFAR Two-Metre Sky Survey -- using the supercomputer JUWELS at the Forschungszentrum Jülich
Authors:
A. Drabent,
M. Hoeft,
A. P. Mechev,
J. B. R. Oonk,
T. W. Shimwell,
F. Sweijen,
A. Danezi,
C. Schrijvers,
C. Manzano,
O. Tsigenov,
R. -J. Dettmar,
M. Brüggen,
D. J. Schwarz
Abstract:
The new generation of high-resolution broad-band radio telescopes, like the Low Frequency Array (LOFAR), produces, depending on the level of compression, between 1 to 10 TB of data per hour after correlation. Such a large amount of scientific data demand powerful computing resources and efficient data handling strategies to be mastered. The LOFAR Two-metre Sky Survey (LoTSS) is a Key Science Proje…
▽ More
The new generation of high-resolution broad-band radio telescopes, like the Low Frequency Array (LOFAR), produces, depending on the level of compression, between 1 to 10 TB of data per hour after correlation. Such a large amount of scientific data demand powerful computing resources and efficient data handling strategies to be mastered. The LOFAR Two-metre Sky Survey (LoTSS) is a Key Science Project (KSP) of the LOFAR telescope. It aims to map the entire northern hemisphere at unprecedented sensitivity and resolution. The survey consist of 3 168 pointings, requiring about 30 PBytes of storage space. As a member of the German Long Wavelength Consortioum (GLOW) the Forschungszentrum Jülich (FSZ) stores in the Long Term Archive (LTA) about 50% of all LoTSS observations conducted to date. In collaboration with SURFsara in Amsterdam we developed service tools that enable the KSP to process LOFAR data stored in the Jülich LTA at the supercomputer JUWELS in an automated and robust fashion. Through our system more than 500 out of 800 existing LoTSS observations have already been processed with the prefactor pipeline. This pipeline calibrates the direction-independent instrumental and ionospheric effects and furthermore reduces the data size significantly. For continuum imaging, this processing pipeline is the standard pipeline that is executed before more advanced processing and image reconstruction methods are applied.
△ Less
Submitted 11 November, 2019; v1 submitted 30 October, 2019;
originally announced October 2019.
-
The LOFAR Two-metre Sky Survey - II. First data release
Authors:
T. W. Shimwell,
C. Tasse,
M. J. Hardcastle,
A. P. Mechev,
W. L. Williams,
P. N. Best,
H. J. A. Röttgering,
J. R. Callingham,
T. J. Dijkema,
F. de Gasperin,
D. N. Hoang,
B. Hugo,
M. Mirmont,
J. B. R. Oonk,
I. Prandoni,
D. Rafferty,
J. Sabater,
O. Smirnov,
R. J. van Weeren,
G. J. White,
M. Atemkeng,
L. Bester,
E. Bonnassieux,
M. Brüggen,
G. Brunetti
, et al. (82 additional authors not shown)
Abstract:
The LOFAR Two-metre Sky Survey (LoTSS) is an ongoing sensitive, high-resolution 120-168MHz survey of the entire northern sky for which observations are now 20% complete. We present our first full-quality public data release. For this data release 424 square degrees, or 2% of the eventual coverage, in the region of the HETDEX Spring Field (right ascension 10h45m00s to 15h30m00s and declination 45…
▽ More
The LOFAR Two-metre Sky Survey (LoTSS) is an ongoing sensitive, high-resolution 120-168MHz survey of the entire northern sky for which observations are now 20% complete. We present our first full-quality public data release. For this data release 424 square degrees, or 2% of the eventual coverage, in the region of the HETDEX Spring Field (right ascension 10h45m00s to 15h30m00s and declination 45$^\circ$00$'$00$''$ to 57$^\circ$00$'$00$''$) were mapped using a fully automated direction-dependent calibration and imaging pipeline that we developed. A total of 325,694 sources are detected with a signal of at least five times the noise, and the source density is a factor of $\sim 10$ higher than the most sensitive existing very wide-area radio-continuum surveys. The median sensitivity is S$_{\rm 144 MHz} = 71\,μ$Jy beam$^{-1}$ and the point-source completeness is 90% at an integrated flux density of 0.45mJy. The resolution of the images is 6$''$ and the positional accuracy is within 0.2$''$. This data release consists of a catalogue containing location, flux, and shape estimates together with 58 mosaic images that cover the catalogued area. In this paper we provide an overview of the data release with a focus on the processing of the LOFAR data and the characteristics of the resulting images. In two accompanying papers we provide the radio source associations and deblending and, where possible, the optical identifications of the radio sources together with the photometric redshifts and properties of the host galaxies. These data release papers are published together with a further $\sim$20 articles that highlight the scientific potential of LoTSS.
△ Less
Submitted 19 November, 2018;
originally announced November 2018.
-
An Automated Scalable Framework for Distributing Radio Astronomy Processing Across Clusters and Clouds
Authors:
A. P. Mechev,
J. B. R. Oonk,
A. Danezi,
T. W. Shimwell,
C. Schrijvers,
H. T. Intema,
A. Plaat,
H. J. A. Röttgering
Abstract:
The Low Frequency Array (LOFAR) radio telescope is an international aperture synthesis radio telescope used to study the Universe at low frequencies. One of the goals of the LOFAR telescope is to conduct deep wide-field surveys. Here we will discuss a framework for the processing of the LOFAR Two Meter Sky Survey (LoTSS). This survey will produce close to 50 PB of data within five years. These dat…
▽ More
The Low Frequency Array (LOFAR) radio telescope is an international aperture synthesis radio telescope used to study the Universe at low frequencies. One of the goals of the LOFAR telescope is to conduct deep wide-field surveys. Here we will discuss a framework for the processing of the LOFAR Two Meter Sky Survey (LoTSS). This survey will produce close to 50 PB of data within five years. These data rates require processing at locations with high-speed access to the archived data. To complete the LoTSS project, the processing software needs to be made portable and moved to clusters with a high bandwidth connection to the data archive. This work presents a framework that makes the LOFAR software portable, and is used to scale out LOFAR data reduction. Previous work was successful in preprocessing LOFAR data on a cluster of isolated nodes. This framework builds upon it and and is currently operational. It is designed to be portable, scalable, automated and general. This paper describes its design and high level operation and the initial results processing LoTSS data.
△ Less
Submitted 10 April, 2018; v1 submitted 1 December, 2017;
originally announced December 2017.
