Euclid preparation

X. The Euclid photometric-redshift challenge

¹ Department of Astronomy, University of Geneva, Ch. d’Écogia 16, 1290 Versoix, Switzerland
² Saudi Information Technology Company, Riyadh 12382, Saudi Arabia
³ King Abdulaziz City for Science and Technology, Riyadh 11442, Saudi Arabia
⁴ Information Engineering, Parks Road, Oxford OX1 3PJ, UK
⁵ School of Physics and Astronomy, Sun Yat-sen University, Guangzhou 519082, Zhuhai Campus, PR China
⁶ INAF-Osservatorio Astronomico di Capodimonte, Via Moiariello 16, 80131 Napoli, Italy
⁷ Department of Physics and Astronomy, University of Missouri, 5110 Rockhill Road, Kansas City, MO 64110, USA
⁸ Department of Physics “E. Pancini”, University Federico II, Via Cinthia 6, 80126 Napoli, Italy
⁹ INFN Section of Naples, Via Cinthia 6, 80126 Napoli, Italy
¹⁰ Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT), Avenida Complutense 40, 28040 Madrid, Spain
¹¹ School of Physics, HH Wills Physics Laboratory, University of Bristol, Tyndall Avenue, Bristol BS8 1TL, UK
¹² Department of Physics, Oxford University, Keble Road, Oxford OX1 3RH, UK
¹³ Aix-Marseille Univ., CNRS, CNES, LAM, Marseille, France
¹⁴ McWilliams Center for Cosmology, Department of Physics, Carnegie Mellon University, Pittsburgh, PA 15213, USA
¹⁵ Center for Astrophysics | Harvard & Smithsonian, 60 Garden St., Cambridge, MA 02138, USA
¹⁶ INAF-Osservatorio Astronomico di Roma, Via Frascati 33, 00078 Monteporzio Catone, Italy
¹⁷ Universitäts-Sternwarte München, Fakultät für Physik, Ludwig-Maximilians-Universität München, Scheinerstraße 1, 81679 München, Germany
¹⁸ Sorbonne Universités, UPMC Univ. Paris 6 et CNRS, UMR 7095, Institut d’Astrophysique de Paris, 98 Bis Bd Arago, 75014 Paris, France
¹⁹ Max Planck Institute for Extraterrestrial Physics, Giessenbachstr. 1, 85748 Garching, Germany
²⁰ INAF-Osservatorio Astronomico di Brera, Via Brera 28, 20122 Milano, Italy
²¹ INAF-Osservatorio di Astrofisica e Scienza dello Spazio di Bologna, Via Piero Gobetti 93/3, 40129 Bologna, Italy
²² SISSA, International School for Advanced Studies, Via Bonomea 265, 34136 Trieste, TS, Italy
²³ INFN, Sezione di Trieste, Via Valerio 2, 34127 Trieste TS, Italy
²⁴ INAF-Osservatorio Astronomico di Trieste, Via G. B. Tiepolo 11, 34131 Trieste, Italy
²⁵ Universidad de la Laguna, 38206 San Cristóbal de La Laguna, Tenerife, Spain
²⁶ Instituto de Astrofísica de Canarias, Calle Vía Làctea s/n, 38204 San Cristóbal de la Laguna, Tenerife, Spain
²⁷ Dipartimento di Fisica e Astronomia, Universitá di Bologna, Via Gobetti 93/2, 40129 Bologna, Italy
²⁸ INFN-Sezione di Bologna, Viale Berti Pichat 6/2, 40127 Bologna, Italy
²⁹ IFPU, Institute for Fundamental Physics of the Universe, Via Beirut 2, 34151 Trieste, Italy
³⁰ INAF-Osservatorio Astrofisico di Torino, Via Osservatorio 20, 10025 Pino Torinese, TO, Italy
³¹ INFN-Sezione di Roma Tre, Via della Vasca Navale 84, 00146 Roma, Italy
³² Department of Mathematics and Physics, Roma Tre University, Via della Vasca Navale 84, 00146 Rome, Italy
³³ Instituto de Astrofísica e Ciências do Espaço, Universidade do Porto, CAUP, Rua das Estrelas, 4150-762 Porto, Portugal
³⁴ Dipartimento di Fisica e Scienze della Terra, Universitá degli Studi di Ferrara, Via Giuseppe Saragat 1, 44122 Ferrara, Italy
³⁵ INAF, Istituto di Radioastronomia, Via Piero Gobetti 101, 40129 Bologna, Italy
³⁶ Institut de Recherche en Astrophysique et Planétologie (IRAP), Université de Toulouse, CNRS, UPS, CNES, 14 Av. Edouard Belin, 31400 Toulouse, France
³⁷ INFN-Sezione di Torino, Via P. Giuria 1, 10125 Torino, Italy
³⁸ Dipartimento di Fisica, Universitá degli Studi di Torino, Via P. Giuria 1, 10125 Torino, Italy
³⁹ Université Côte d’Azur, Observatoire de la Côte d’Azur, CNRS, Laboratoire Lagrange, Bd de l’Observatoire, CS 34229, 06304 Nice Cedex 4, France
⁴⁰ INAF-IASF Milano, Via Alfonso Corti 12, 20133 Milano, Italy
⁴¹ Institut de Física d’Altes Energies (IFAE), The Barcelona Institute of Science and Technology, Campus UAB, 08193 Bellaterra, Barcelona, Spain
⁴² Instituto de Astrofísica e Ciências do Espaço, Faculdade de Ciências, Universidade de Lisboa, Tapada da Ajuda, 1349-018 Lisboa, Portugal
⁴³ Institute of Space Sciences (ICE, CSIC), Campus UAB, Carrer de Can Magrans, s/n, 08193 Barcelona, Spain
⁴⁴ Institut d’Estudis Espacials de Catalunya (IEEC), 08034 Barcelona, Spain
⁴⁵ AIM, CEA, CNRS, Université Paris-Saclay, Université Paris Diderot, Sorbonne Paris Cité, 91191 Gif-sur-Yvette, France
⁴⁶ Observatoire de Sauverny, Ecole Polytechnique Fédérale de Lau- sanne, 1290 Versoix, Switzerland
⁴⁷ INAF-Osservatorio Astrofisico di Arcetri, Largo E. Fermi 5, 50125 Firenze, Italy
⁴⁸ Centre National d’Etudes Spatiales, Toulouse, France
⁴⁹ Institute for Astronomy, University of Edinburgh, Royal Observatory, Blackford Hill, Edinburgh EH9 3HJ, UK
⁵⁰ University of Nottingham, University Park, Nottingham NG7 2RD, UK
⁵¹ European Space Agency/ESRIN, Largo Galileo Galilei 1, 00044 Frascati, Roma, Italy
⁵² ESAC/ESA, Camino Bajo del Castillo, s/n, Urb. Villafranca del Castillo, 28692 Villanueva de la Cañada, Madrid, Spain
⁵³ Univ. Lyon, Univ. Claude Bernard Lyon 1, CNRS/IN2P3, IP2I Lyon, UMR 5822, 69622 Villeurbanne, France
⁵⁴ University of Lyon, UCB Lyon 1, CNRS/IN2P3, IUF, IP2I, Lyon, France
⁵⁵ Departamento de Física, Faculdade de Ciências, Universidade de Lisboa, Edifício C8, Campo Grande 1749-016, Lisboa, Portugal
⁵⁶ Instituto de Astrofísica e Ciências do Espaço, Faculdade de Ciências, Universidade de Lisboa, Campo Grande, 1749-016 Lisboa, Portugal
⁵⁷ Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France
⁵⁸ Department of Physics & Astronomy, University of Sussex, Brighton BN1 9QH, UK
⁵⁹ Astrophysics Group, Blackett Laboratory, Imperial College London, London SW7 2AZ, UK
⁶⁰ INFN-Bologna, Via Irnerio 46, 40126 Bologna, Italy
⁶¹ Institut de Physique Nucléaire de Lyon, 4, Rue Enrico Fermi, 69622 Villeurbanne Cedex, France
⁶² Aix-Marseille Univ., CNRS/IN2P3, CPPM, Marseille, France
⁶³ Department of Physics, University of Helsinki, PO Box 64, 00014 Helsinki, Finland
⁶⁴ Department of Physics and Helsinki Institute of Physics, University of Helsinki, Gustaf Hällströmin Katu 2, 00014 Helsinki, Finland
⁶⁵ Dipartimento di Fisica “Aldo Pontremoli”, Universitá degli Studi di Milano, Via Celoria 16, 20133 Milano, Italy
⁶⁶ INFN-Sezione di Milano, Via Celoria 16, 20133 Milano, Italy
⁶⁷ Mullard Space Science Laboratory, University College London, Holmbury St Mary, Dorking, Surrey RH5 6NT, UK
⁶⁸ Institute of Theoretical Astrophysics, University of Oslo, PO Box 1029, Blindern 0315, Oslo, Norway
⁶⁹ Jet Propulsion Laboratory, California Institute of Technology, 4800 Oak Grove Drive, Pasadena, CA 91109, USA
⁷⁰ von Hoerner & Sulger GmbH, SchloßPlatz 8, 68723 Schwetzingen, Germany
⁷¹ Max-Planck-Institut für Astronomie, Königstuhl 17, 69117 Heidelberg, Germany
⁷² Institut d’Astrophysique de Paris, 98Bis Boulevard Arago, 75014 Paris, France
⁷³ Université de Genève, Département de Physique Théorique and Centre for Astroparticle Physics, 24 Quai Ernest-Ansermet, 1211 Genève 4, Switzerland
⁷⁴ NOVA Optical Infrared Instrumentation Group at ASTRON, Oude Hoogeveensedijk 4, 7991 PD Dwingeloo, The Netherlands
⁷⁵ Argelander-Institut für Astronomie, Universität Bonn, Auf dem Hügel 71, 53121 Bonn, Germany
⁷⁶ Institute for Computational Cosmology, Department of Physics, Durham University, South Road, Durham, DH1 3LE, UK
⁷⁷ Institut für Theoretische Physik, University of Heidelberg, Philosophenweg 16, 69120 Heidelberg, Germany
⁷⁸ Zentrum für Astronomie, Universität Heidelberg, Philosophenweg 12, 69120 Heidelberg, Germany
⁷⁹ Université Côte d’Azur, Observatoire de la Côte d’Azur, CNRS, Laboratoire Lagrange, France
⁸⁰ INAF-IASF Bologna, Via Piero Gobetti 101, 40129 Bologna, Italy
⁸¹ Université de Paris, 75013 Paris, France
⁸² LERMA, Observatoire de Paris, PSL Research University, CNRS, Sorbonne Université, 75014 Paris, France
⁸³ Space Science Data Center, Italian Space Agency, Via del Politecnico snc, 00133 Roma, Italy
⁸⁴ Institute of Space Science, Bucharest 077125, Romania
⁸⁵ Institute for Computational Science, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland
⁸⁶ INFN-Padova, Via Marzolo 8, 35131 Padova, Italy
⁸⁷ Dipartimento di Fisica e Astronomia “G.Galilei”, Universitá di Padova, Via Marzolo 8, 35131 Padova, Italy
⁸⁸ Departamento de Física, FCFM, Universidad de Chile, Blanco Encalada 2008 Santiago, Chile
⁸⁹ Universidad Politécnica de Cartagena, Departamento de Electrónica y Tecnología de Computadoras, 30202 Cartagena, Spain
⁹⁰ Infrared Processing and Analysis Center, California Institute of Technology, Pasadena, CA 91125, USA
⁹¹ Jodrell Bank Centre for Astrophysics, School of Physics and Astronomy, University of Manchester, Oxford Road, Manchester M13 9PL, UK
⁹² Department of Physics and Astronomy, University College London, Gower Street, London WC1E 6BT, UK

Received: 11 September 2020
Accepted: 20 October 2020

Abstract

Forthcoming large photometric surveys for cosmology require precise and accurate photometric redshift (photo-z) measurements for the success of their main science objectives. However, to date, no method has been able to produce photo-zs at the required accuracy using only the broad-band photometry that those surveys will provide. An assessment of the strengths and weaknesses of current methods is a crucial step in the eventual development of an approach to meet this challenge. We report on the performance of 13 photometric redshift code single value redshift estimates and redshift probability distributions (PDZs) on a common set of data, focusing particularly on the 0.2 − 2.6 redshift range that the Euclid mission will probe. We designed a challenge using emulated Euclid data drawn from three photometric surveys of the COSMOS field. The data was divided into two samples: one calibration sample for which photometry and redshifts were provided to the participants; and the validation sample, containing only the photometry to ensure a blinded test of the methods. Participants were invited to provide a redshift single value estimate and a PDZ for each source in the validation sample, along with a rejection flag that indicates the sources they consider unfit for use in cosmological analyses. The performance of each method was assessed through a set of informative metrics, using cross-matched spectroscopic and highly-accurate photometric redshifts as the ground truth. We show that the rejection criteria set by participants are efficient in removing strong outliers, that is to say sources for which the photo-z deviates by more than 0.15(1 + z) from the spectroscopic-redshift (spec-z). We also show that, while all methods are able to provide reliable single value estimates, several machine-learning methods do not manage to produce useful PDZs. We find that no machine-learning method provides good results in the regions of galaxy color-space that are sparsely populated by spectroscopic-redshifts, for example z >  1. However they generally perform better than template-fitting methods at low redshift (z <  0.7), indicating that template-fitting methods do not use all of the information contained in the photometry. We introduce metrics that quantify both photo-z precision and completeness of the samples (post-rejection), since both contribute to the final figure of merit of the science goals of the survey (e.g., cosmic shear from Euclid). Template-fitting methods provide the best results in these metrics, but we show that a combination of template-fitting results and machine-learning results with rejection criteria can outperform any individual method. On this basis, we argue that further work in identifying how to best select between machine-learning and template-fitting approaches for each individual galaxy should be pursued as a priority.