PHAT: PHoto-z Accuracy Testing^⋆

¹ Leiden Observatory, Leiden University, Niels Bohrweg 2, 2333CA Leiden, The Netherlands
e-mail: hendrik@strw.leidenuniv.nl
² Canada-France-Hawaii Telescope Corporation, Kamuela, HI 96743, USA
³ Spitzer Science Center, 314-6, California Institute of Technology, 1201 E. California Blvd, Pasadena, CA, 91125, USA
⁴ Jet Propulsion Laboratory, California Institute of Technology, MS 169-327, Pasadena, CA 91109, USA
⁵ Department of Physics, University of Oxford, DWB, Keble Road, Oxford, OX1 3RH, UK
⁶ Department of Physics and Astronomy, University College London, Gower Street, London WC1E 6BT, UK
⁷ Department of Astronomy, The Ohio State University, 4055 McPherson Lab, 140 W. 18th Avenue, Columbus, OH 43210, USA
⁸ Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge, CB3 0HA, UK
⁹ Instituto de Astrofísica de Andalucía (CSIC), Apdo. 3044, 18008 Granada, Spain
¹⁰ Department of Astronomy, Yale University, New Haven, CT 06520-8101, USA
¹¹ Department of Physics and Astronomy, Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA
¹² Department of Computer Science, Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA
¹³ Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD 21218, USA
¹⁴ Department of Physics, Institute of Astronomy, ETH Zürich, Wolfgang-Pauli-Strasse 16, 8093 Zürich, Switzerland
¹⁵ Department of Physics, University of Michigan, Ann Arbor, Michigan 48109, USA
¹⁶ Department of Physics and Astronomy, University of Waterloo, 200 University Avenue West, Waterloo, Ontario, N2L 3G1, Canada
¹⁷ Laboratoire d’Astrophysique de Marseille, CNRS-Université d’Aix-Marseille, 38 rue Frédéric Joliot-Curie, 13388 Marseille Cedex 13, France
¹⁸ Centre for Astrophysics Research, University of Hertfordshire, College Lane, Hatfield AL10 9AB, UK
¹⁹ Department of Astronomy, University of Wisconsin-Madison, 475 N Charter St., Madison, WI 53706, USA
²⁰ Centre for Astrophysics & Supercomputing, Swinburne University of Technology, PO Box 218, Hawthorn, VIC 3122, Australia
²¹ Institut d’Estudis Andorrans, Avda Rocafort 21–23, AD 600 Sant Julià de Lòria, Andorra
²² Department of Physics of Complex Systems, Eötvös Loránd University, Pf. 32, 1518 Budapest, Hungary
²³ Physics Department, University of California, 1 Shields Avenue, Davis, CA 95616, USA
²⁴ Kavli Institute for Particle Astrophysics and Cosmology, SLAC National Accelerator Laboratory, Menlo Park, CA 94025, USA

Received: 29 April 2010
Accepted: 23 July 2010

Abstract

Context. Photometric redshifts (photo-z’s) have become an essential tool in extragalactic astronomy. Many current and upcoming observing programmes require great accuracy of photo-z’s to reach their scientific goals.

Aims. Here we introduce PHAT, the PHoto-z Accuracy Testing programme, an international initiative to test and compare different methods of photo-z estimation.

Methods. Two different test environments are set up, one (PHAT0) based on simulations to test the basic functionality of the different photo-z codes, and another one (PHAT1) based on data from the GOODS survey including 18-band photometry and  ~2000 spectroscopic redshifts.

Results. The accuracy of the different methods is expressed and ranked by the global photo-z bias, scatter, and outlier rates. While most methods agree very well on PHAT0 there are differences in the handling of the Lyman-α forest for higher redshifts. Furthermore, different methods produce photo-z scatters that can differ by up to a factor of two even in this idealised case. A larger spread in accuracy is found for PHAT1. Few methods benefit from the addition of mid-IR photometry. The accuracy of the other methods is unaffected or suffers when IRAC data are included. Remaining biases and systematic effects can be explained by shortcomings in the different template sets (especially in the mid-IR) and the use of priors on the one hand and an insufficient training set on the other hand. Some strategies to overcome these problems are identified by comparing the methods in detail. Scatters of 4–8% in Δz / (1 + z) were obtained, consistent with other studies. However, somewhat larger outlier rates (>7.5% with Δz / (1 + z) > 0.15;  > 4.5% after cleaning) are found for all codes that can only partly be explained by AGN or issues in the photometry or the spec-z catalogue. Some outliers were probably missed in comparisons of photo-z’s to other, less complete spectroscopic surveys in the past. There is a general trend that empirical codes produce smaller biases than template-based codes.

Conclusions. The systematic, quantitative comparison of different photo-z codes presented here is a snapshot of the current state-of-the-art of photo-z estimation and sets a standard for the assessment of photo-z accuracy in the future. The rather large outlier rates reported here for PHAT1 on real data should be investigated further since they are most probably also present (and possibly hidden) in many other studies. The test data sets are publicly available and can be used to compare new, upcoming methods to established ones and help in guiding future photo-z method development.