The Dark Energy Survey Data Processing and Calibration System
Authors:
Joseph J. Mohr,
Robert Armstrong,
Emmanuel Bertin,
Gregory E. Daues,
Shantanu Desai,
Michelle Gower,
Robert Gruendl,
William Hanlon,
Nikolay Kuropatkin,
Huan Lin,
John Marriner,
Don Petravick,
Ignacio Sevilla,
Molly Swanson,
Todd Tomashek,
Douglas Tucker,
Brian Yanny,
the Dark Energy Survey Collaboration
Abstract:
The Dark Energy Survey (DES) is a 5000 deg2 grizY survey reaching characteristic photometric depths of 24th magnitude (10 sigma) and enabling accurate photometry and morphology of objects ten times fainter than in SDSS. Preparations for DES have included building a dedicated 3 deg2 CCD camera (DECam), upgrading the existing CTIO Blanco 4m telescope and developing a new high performance computing (…
▽ More
The Dark Energy Survey (DES) is a 5000 deg2 grizY survey reaching characteristic photometric depths of 24th magnitude (10 sigma) and enabling accurate photometry and morphology of objects ten times fainter than in SDSS. Preparations for DES have included building a dedicated 3 deg2 CCD camera (DECam), upgrading the existing CTIO Blanco 4m telescope and developing a new high performance computing (HPC) enabled data management system (DESDM).
The DESDM system will be used for processing, calibrating and serving the DES data. The total data volumes are high (~2PB), and so considerable effort has gone into designing an automated processing and quality control system. Special purpose image detrending and photometric calibration codes have been developed to meet the data quality requirements, while survey astrometric calibration, coaddition and cataloging rely on new extensions of the AstrOmatic codes which now include tools for PSF modeling, PSF homogenization, PSF corrected model fitting cataloging and joint model fitting across multiple input images.
The DESDM system has been deployed on dedicated development clusters and HPC systems in the US and Germany. An extensive program of testing with small rapid turn-around and larger campaign simulated datasets has been carried out. The system has also been tested on large real datasets, including Blanco Cosmology Survey data from the Mosaic2 camera. In Fall 2012 the DESDM system will be used for DECam commissioning, and, thereafter, the system will go into full science operations.
△ Less
Submitted 13 July, 2012;
originally announced July 2012.
The Dark Energy Survey Data Management System
Authors:
I. Sevilla,
R. Armstrong,
E. Bertin,
A. Carlson,
G. Daues,
S. Desai,
M. Gower,
R. Gruendl,
W. Hanlon,
M. Jarvis,
R. Kessler,
N. Kuropatkin,
H. Lin,
J. Marriner,
J. Mohr,
D. Petravick,
E. Sheldon,
M. E. C. Swanson,
T. Tomashek,
D. Tucker,
Y. Yang,
B. Yanny
Abstract:
The Dark Energy Survey (DES) is a project with the goal of building, installing and exploiting a new 74 CCD-camera at the Blanco telescope, in order to study the nature of cosmic acceleration. It will cover 5000 square degrees of the southern hemisphere sky and will record the positions and shapes of 300 million galaxies up to redshift 1.4. The survey will be completed using 525 nights during a 5-…
▽ More
The Dark Energy Survey (DES) is a project with the goal of building, installing and exploiting a new 74 CCD-camera at the Blanco telescope, in order to study the nature of cosmic acceleration. It will cover 5000 square degrees of the southern hemisphere sky and will record the positions and shapes of 300 million galaxies up to redshift 1.4. The survey will be completed using 525 nights during a 5-year period starting in 2012. About O(1 TB) of raw data will be produced every night, including science and calibration images. The DES data management system has been designed for the processing, calibration and archiving of these data. It is being developed by collaborating DES institutions, led by NCSA. In this contribution, we describe the basic functions of the system, what kind of scientific codes are involved and how the Data Challenge process works, to improve simultaneously the Data Management system algorithms and the Science Working Group analysis codes.
△ Less
Submitted 30 September, 2011;
originally announced September 2011.
