Euclid Quick Data Release (Q1)

XIII. Exploring galaxy properties with a multi-modal foundation model

¹ Instituto de Astrofísica de Canarias (IAC); Departamento de Astrofísica, Universidad de La Laguna (ULL), 38200 La Laguna, Tenerife, Spain
² Institute of Space Sciences (ICE, CSIC), Campus UAB, Carrer de Can Magrans, s/n, 08193 Barcelona, Spain
³ Instituto de Astrofísica de Canarias, Vía Láctea, 38205 La Laguna, Tenerife, Spain
⁴ Université PSL, Observatoire de Paris, Sorbonne Université, CNRS, LERMA, 75014 Paris, France
⁵ Université Paris-Cité, 5 Rue Thomas Mann, 75013 Paris, France
⁶ School of Physics, Astronomy and Mathematics, University of Hertfordshire, College Lane, Hatfield AL10 9AB, UK
⁷ UniverseTBD, https://universetbd.org
⁸ Aspia Space, Falmouth TR10 9TA, UK
⁹ Instituto de Astrofísica de Andalucía, CSIC, Glorieta de la Astronomía, 18080 Granada, Spain
¹⁰ Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM, 91191 Gif-sur-Yvette, France
¹¹ Center for Computational Astrophysics, Flatiron Institute, 162 5th Avenue, 10010 New York, NY, USA
¹² Center for Cosmology and Particle Physics, Department of Physics, New York University, New York, NY 10003, USA
¹³ Princeton University, Princeton, NJ 08540, USA
¹⁴ Universidad de La Laguna, Departamento de Astrofísica, 38206 La Laguna, Tenerife, Spain
¹⁵ Faculdade de Ciências da Universidade do Porto, Rua do Campo de Alegre, 4150-007 Porto, Portugal
¹⁶ Instituto de Astrofísica e Ciências do Espaço, Universidade do Porto, CAUP, Rua das Estrelas, PT4150-762 Porto, Portugal
¹⁷ Instituto de Física de Cantabria, Edificio Juan Jordá, Avenida de los Castros, 39005 Santander, Spain
¹⁸ Leiden Observatory, Leiden University, Einsteinweg 55, 2333 CC Leiden, The Netherlands
¹⁹ Kapteyn Astronomical Institute, University of Groningen, PO Box 800, 9700 AV Groningen, The Netherlands
²⁰ INAF-Osservatorio Astronomico di Capodimonte, Via Moiariello 16, 80131 Napoli, Italy
²¹ Max Planck Institute for Extraterrestrial Physics, Giessenbachstr. 1, 85748 Garching, Germany
²² Institut d’Estudis Espacials de Catalunya (IEEC), Edifici RDIT, Campus UPC, 08860 Castelldefels, Barcelona, Spain
²³ School of Physics, HH Wills Physics Laboratory, University of Bristol, Tyndall Avenue, Bristol BS8 1TL, UK
²⁴ Centro de Estudios de Física del Cosmos de Aragón (CEFCA), Plaza San Juan, 1, planta 2, 44001 Teruel, Spain
²⁵ Université Paris-Saclay, CNRS, Institut d’astrophysique spatiale, 91405 Orsay, France
²⁶ ESAC/ESA, Camino Bajo del Castillo, s/n., Urb. Villafranca del Castillo, 28692 Villanueva de la Cañada, Madrid, Spain
²⁷ School of Mathematics and Physics, University of Surrey, Guildford, Surrey GU2 7XH, UK
²⁸ INAF-Osservatorio Astronomico di Brera, Via Brera 28, 20122 Milano, Italy
²⁹ INAF-Osservatorio di Astrofisica e Scienza dello Spazio di Bologna, Via Piero Gobetti 93/3, 40129 Bologna, Italy
³⁰ IFPU, Institute for Fundamental Physics of the Universe, Via Beirut 2, 34151 Trieste, Italy
³¹ INAF-Osservatorio Astronomico di Trieste, Via G. B. Tiepolo 11, 34143 Trieste, Italy
³² INFN, Sezione di Trieste, Via Valerio 2, 34127 Trieste TS, Italy
³³ SISSA, International School for Advanced Studies, Via Bonomea 265, 34136 Trieste TS, Italy
³⁴ Dipartimento di Fisica e Astronomia, Università di Bologna, Via Gobetti 93/2, 40129 Bologna, Italy
³⁵ INFN-Sezione di Bologna, Viale Berti Pichat 6/2, 40127 Bologna, Italy
³⁶ Space Science Data Center, Italian Space Agency, Via del Politecnico snc, 00133 Roma, Italy
³⁷ Dipartimento di Fisica, Università di Genova, Via Dodecaneso 33, 16146 Genova, Italy
³⁸ INFN-Sezione di Genova, Via Dodecaneso 33, 16146 Genova, Italy
³⁹ Department of Physics “E. Pancini”, University Federico II, Via Cinthia 6, 80126 Napoli, Italy
⁴⁰ Dipartimento di Fisica, Università degli Studi di Torino, Via P. Giuria 1, 10125 Torino, Italy
⁴¹ INFN-Sezione di Torino, Via P. Giuria 1, 10125 Torino, Italy
⁴² INAF-Osservatorio Astrofisico di Torino, Via Osservatorio 20, 10025 Pino Torinese (TO), Italy
⁴³ European Space Agency/ESTEC, Keplerlaan 1, 2201 AZ Noordwijk, The Netherlands
⁴⁴ Institute Lorentz, Leiden University, Niels Bohrweg 2, 2333 CA Leiden, The Netherlands
⁴⁵ INAF-IASF Milano, Via Alfonso Corti 12, 20133 Milano, Italy
⁴⁶ Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT), Avenida Complutense 40, 28040 Madrid, Spain
⁴⁷ Port d’Informació Científica, Campus UAB, C. Albareda s/n, 08193 Bellaterra (Barcelona), Spain
⁴⁸ Institute for Theoretical Particle Physics and Cosmology (TTK), RWTH Aachen University, 52056 Aachen, Germany
⁴⁹ INAF-Osservatorio Astronomico di Roma, Via Frascati 33, 00078 Monteporzio Catone, Italy
⁵⁰ INFN section of Naples, Via Cinthia 6, 80126 Napoli, Italy
⁵¹ Institute for Astronomy, University of Hawaii, 2680 Woodlawn Drive, Honolulu, HI 96822, USA
⁵² Dipartimento di Fisica e Astronomia “Augusto Righi” – Alma Mater Studiorum Università di Bologna, Viale Berti Pichat 6/2, 40127 Bologna, Italy
⁵³ Institute for Astronomy, University of Edinburgh, Royal Observatory, Blackford Hill, Edinburgh EH9 3HJ, UK
⁵⁴ Jodrell Bank Centre for Astrophysics, Department of Physics and Astronomy, University of Manchester, Oxford Road, Manchester M13 9PL, UK
⁵⁵ European Space Agency/ESRIN, Largo Galileo Galilei 1, 00044 Frascati, Roma, Italy
⁵⁶ Université Claude Bernard Lyon 1, CNRS/IN2P3, IP2I Lyon, UMR 5822, Villeurbanne F-69100, France
⁵⁷ Institut de Ciències del Cosmos (ICCUB), Universitat de Barcelona (IEEC-UB), Martí i Franquès 1, 08028 Barcelona, Spain
⁵⁸ Institució Catalana de Recerca i Estudis Avançats (ICREA), Passeig de Lluís Companys 23, 08010 Barcelona, Spain
⁵⁹ UCB Lyon 1, CNRS/IN2P3, IUF, IP2I Lyon, 4 rue Enrico Fermi, 69622 Villeurbanne, France
⁶⁰ Mullard Space Science Laboratory, University College London, Holmbury St Mary, Dorking, Surrey RH5 6NT, UK
⁶¹ Departamento de Física, Faculdade de Ciências, Universidade de Lisboa, Edifício C8, Campo Grande, PT1749-016 Lisboa, Portugal
⁶² Instituto de Astrofísica e Ciências do Espaço, Faculdade de Ciências, Universidade de Lisboa, Campo Grande, 1749-016 Lisboa, Portugal
⁶³ Department of Astronomy, University of Geneva, ch. d’Ecogia 16, 1290 Versoix, Switzerland
⁶⁴ INAF-Istituto di Astrofisica e Planetologia Spaziali, Via del Fosso del Cavaliere, 100, 00100 Roma, Italy
⁶⁵ INFN-Padova, Via Marzolo 8, 35131 Padova, Italy
⁶⁶ Aix-Marseille Université, CNRS/IN2P3, CPPM, Marseille, France
⁶⁷ INFN-Bologna, Via Irnerio 46, 40126 Bologna, Italy
⁶⁸ Universitäts-Sternwarte München, Fakultät für Physik, Ludwig-Maximilians-Universität München, Scheinerstrasse 1, 81679 München, Germany
⁶⁹ INAF-Osservatorio Astronomico di Padova, Via dell’Osservatorio 5, 35122 Padova, Italy
⁷⁰ NRC Herzberg, 5071 West Saanich Rd, Victoria, BC V9E 2E7, Canada
⁷¹ Institute of Theoretical Astrophysics, University of Oslo, P.O. Box 1029 Blindern, 0315 Oslo, Norway
⁷² Jet Propulsion Laboratory, California Institute of Technology, 4800 Oak Grove Drive, Pasadena, CA 91109, USA
⁷³ Department of Physics, Lancaster University, Lancaster LA1 4YB, UK
⁷⁴ Felix Hormuth Engineering, Goethestr. 17, 69181 Leimen, Germany
⁷⁵ Technical University of Denmark, Elektrovej 327, 2800 Kgs. Lyngby, Denmark
⁷⁶ Cosmic Dawn Center (DAWN), Denmark
⁷⁷ Max-Planck-Institut für Astronomie, Königstuhl 17, 69117 Heidelberg, Germany
⁷⁸ NASA Goddard Space Flight Center, Greenbelt, MD 20771, USA
⁷⁹ Department of Physics and Helsinki Institute of Physics, Gustaf Hällströmin katu 2, 00014 University of Helsinki, Finland
⁸⁰ Université de Genève, Département de Physique Théorique and Centre for Astroparticle Physics, 24 quai Ernest-Ansermet, CH-1211 Genève 4, Switzerland
⁸¹ Department of Physics, P.O. Box 64, 00014 University of Helsinki, Finland
⁸² Helsinki Institute of Physics, Gustaf Hällströmin katu 2 University of Helsinki, Helsinki, Finland
⁸³ Centre de Calcul de l’IN2P3/CNRS, 21 avenue Pierre de Coubertin 69627 Villeurbanne Cedex, France
⁸⁴ Laboratoire d’etude de l’Univers et des phenomenes eXtremes, Observatoire de Paris, Université PSL, Sorbonne Université, CNRS, 92190 Meudon, France
⁸⁵ Aix-Marseille Université, CNRS, CNES, LAM, Marseille, France
⁸⁶ SKA Observatory, Jodrell Bank, Lower Withington, Macclesfield, Cheshire SK11 9FT, UK
⁸⁷ Dipartimento di Fisica “Aldo Pontremoli”, Università degli Studi di Milano, Via Celoria 16, 20133 Milano, Italy
⁸⁸ INFN-Sezione di Milano, Via Celoria 16, 20133 Milano, Italy
⁸⁹ University of Applied Sciences and Arts of Northwestern Switzerland, School of Computer Science, 5210 Windisch, Switzerland
⁹⁰ Universität Bonn, Argelander-Institut für Astronomie, Auf dem Hügel 71, 53121 Bonn, Germany
⁹¹ INFN-Sezione di Roma, Piazzale Aldo Moro, 2 – c/o Dipartimento di Fisica, Edificio G. Marconi, 00185 Roma, Italy
⁹² Dipartimento di Fisica e Astronomia “Augusto Righi” – Alma Mater Studiorum Università di Bologna, Via Piero Gobetti 93/2, 40129 Bologna, Italy
⁹³ Department of Physics, Institute for Computational Cosmology, Durham University, South Road, Durham DH1 3LE, UK
⁹⁴ Université Côte d’Azur, Observatoire de la Côte d’Azur, CNRS, Laboratoire Lagrange, Bd de l’Observatoire, CS 34229, 06304 Nice cedex 4, France
⁹⁵ Institut d’Astrophysique de Paris, UMR 7095, CNRS, and Sorbonne Université, 98 bis boulevard Arago, 75014 Paris, France
⁹⁶ Université Paris Cité, CNRS, Astroparticule et Cosmologie, 75013 Paris, France
⁹⁷ CNRS-UCB International Research Laboratory, Centre Pierre Binétruy, IRL2007, CPB-IN2P3, Berkeley, USA
⁹⁸ University of Applied Sciences and Arts of Northwestern Switzerland, School of Engineering, 5210 Windisch, Switzerland
⁹⁹ Institut d’Astrophysique de Paris, 98bis Boulevard Arago, 75014 Paris, France
¹⁰⁰ Institute of Physics, Laboratory of Astrophysics, Ecole Polytechnique Fédérale de Lausanne (EPFL), Observatoire de Sauverny, 1290 Versoix, Switzerland
¹⁰¹ Aurora Technology for European Space Agency (ESA), Camino bajo del Castillo, s/n, Urbanizacion Villafranca del Castillo, Villanueva de la Cañada, 28692 Madrid, Spain
¹⁰² Institut de Física d’Altes Energies (IFAE), The Barcelona Institute of Science and Technology, Campus UAB, 08193 Bellaterra (Barcelona), Spain
¹⁰³ School of Mathematics, Statistics and Physics, Newcastle University, Herschel Building, Newcastle-upon-Tyne, NE1 7RU, UK
¹⁰⁴ DARK, Niels Bohr Institute, University of Copenhagen, Jagtvej 155, 2200 Copenhagen, Denmark
¹⁰⁵ Waterloo Centre for Astrophysics, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada
¹⁰⁶ Department of Physics and Astronomy, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada
¹⁰⁷ Perimeter Institute for Theoretical Physics, Waterloo, Ontario N2L 2Y5, Canada
¹⁰⁸ Centre National d’Etudes Spatiales – Centre spatial de Toulouse, 18 avenue Edouard Belin, 31401 Toulouse Cedex 9, France
¹⁰⁹ Institute of Space Science, Str. Atomistilor, nr. 409Măgurele, Ilfov 077125, Romania
¹¹⁰ Dipartimento di Fisica e Astronomia “G. Galilei”, Università di Padova, Via Marzolo 8, 35131 Padova, Italy
¹¹¹ Institut für Theoretische Physik, University of Heidelberg, Philosophenweg 16, 69120 Heidelberg, Germany
¹¹² Institut de Recherche en Astrophysique et Planétologie (IRAP), Université de Toulouse, CNRS, UPS, CNES, 14 Av. Edouard Belin, 31400 Toulouse, France
¹¹³ Université St Joseph; Faculty of Sciences, Beirut, Lebanon
¹¹⁴ Departamento de Física, FCFM, Universidad de Chile, Blanco Encalada 2008, Santiago, Chile
¹¹⁵ Universität Innsbruck, Institut für Astro- und Teilchenphysik, Technikerstr. 25/8, 6020 Innsbruck, Austria
¹¹⁶ Satlantis, University Science Park, Sede Bld 48940, Leioa-Bilbao, Spain
¹¹⁷ Instituto de Astrofísica e Ciências do Espaço, Faculdade de Ciências, Universidade de Lisboa, Tapada da Ajuda, 1349-018 Lisboa, Portugal
¹¹⁸ Cosmic Dawn Center (DAWN)
¹¹⁹ Niels Bohr Institute, University of Copenhagen, Jagtvej 128, 2200 Copenhagen, Denmark
¹²⁰ Universidad Politécnica de Cartagena, Departamento de Electrónica y Tecnología de Computadoras, Plaza del Hospital 1, 30202 Cartagena, Spain
¹²¹ Infrared Processing and Analysis Center, California Institute of Technology, Pasadena, CA 91125, USA
¹²² Dipartimento di Fisica e Scienze della Terra, Università degli Studi di Ferrara, Via Giuseppe Saragat 1, 44122 Ferrara, Italy
¹²³ Istituto Nazionale di Fisica Nucleare, Sezione di Ferrara, Via Giuseppe Saragat 1, 44122 Ferrara, Italy
¹²⁴ INAF, Istituto di Radioastronomia, Via Piero Gobetti 101, 40129 Bologna, Italy
¹²⁵ Department of Physics, Oxford University, Keble Road, Oxford OX1 3RH, UK
¹²⁶ INAF – Osservatorio Astronomico di Brera, Via Emilio Bianchi 46, 23807 Merate, Italy
¹²⁷ INAF-Osservatorio Astronomico di Brera, Via Brera 28, 20122 Milano, Italy, and INFN-Sezione di Genova, Via Dodecaneso 33, 16146 Genova, Italy
¹²⁸ ICL, Junia, Université Catholique de Lille, LITL, 59000 Lille, France
¹²⁹ ICSC – Centro Nazionale di Ricerca in High Performance Computing, Big Data e Quantum Computing, Via Magnanelli 2, Bologna, Italy
¹³⁰ Instituto de Física Teórica UAM-CSIC, Campus de Cantoblanco, 28049 Madrid, Spain
¹³¹ CERCA/ISO, Department of Physics, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106, USA
¹³² Technical University of Munich, TUM School of Natural Sciences, Physics Department, James-Franck-Str. 1, 85748 Garching, Germany
¹³³ Max-Planck-Institut für Astrophysik, Karl-Schwarzschild-Str. 1, 85748 Garching, Germany
¹³⁴ Laboratoire Univers et Théorie, Observatoire de Paris, Université PSL, Université Paris Cité, CNRS, 92190 Meudon, France
¹³⁵ Departamento de Física Fundamental. Universidad de Salamanca, Plaza de la Merced s/n, 37008 Salamanca, Spain
¹³⁶ Université de Strasbourg, CNRS, Observatoire astronomique de Strasbourg, UMR 7550, 67000 Strasbourg, France
¹³⁷ Center for Data-Driven Discovery, Kavli IPMU (WPI), UTIAS, The University of Tokyo, Kashiwa, Chiba 277-8583, Japan
¹³⁸ California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125, USA
¹³⁹ Department of Physics & Astronomy, University of California Irvine, Irvine CA 92697, USA
¹⁴⁰ Department of Mathematics and Physics E. De Giorgi, University of Salento, Via per Arnesano, CP-I93, 73100 Lecce, Italy
¹⁴¹ INFN, Sezione di Lecce, Via per Arnesano, CP-193, 73100 Lecce, Italy
¹⁴² INAF-Sezione di Lecce, c/o Dipartimento Matematica e Fisica, Via per Arnesano, 73100 Lecce, Italy
¹⁴³ Departamento Física Aplicada, Universidad Politécnica de Cartagena, Campus Muralla del Mar, 30202 Cartagena, Murcia, Spain
¹⁴⁴ CEA Saclay, DFR/IRFU, Service d’Astrophysique, Bat. 709, 91191 Gif-sur-Yvette, France
¹⁴⁵ Institute of Cosmology and Gravitation, University of Portsmouth, Portsmouth PO1 3FX, UK
¹⁴⁶ Department of Computer Science, Aalto University, PO Box 15400 Espoo FI-00 076, Finland
¹⁴⁷ Instituto de Astrofísica de Canarias, c/ Via Lactea s/n, La Laguna 38200, Spain. Departamento de Astrofísica de la Universidad de La Laguna, Avda. Francisco Sanchez, La Laguna, 38200, Spain
¹⁴⁸ Caltech/IPAC, 1200 E. California Blvd., Pasadena, CA 91125, USA
¹⁴⁹ Ruhr University Bochum, Faculty of Physics and Astronomy, Astronomical Institute (AIRUB), German Centre for Cosmological Lensing (GCCL), 44780 Bochum, Germany
¹⁵⁰ Department of Physics and Astronomy, Vesilinnantie 5, 20014 University of Turku, Finland
¹⁵¹ Serco for European Space Agency (ESA), Camino bajo del Castillo, s/n, Urbanizacion Villafranca del Castillo, Villanueva de la Cañada, 28692 Madrid, Spain
¹⁵² ARC Centre of Excellence for Dark Matter Particle Physics, Melbourne, Australia
¹⁵³ Centre for Astrophysics & Supercomputing, Swinburne University of Technology, Hawthorn, Victoria 3122, Australia
¹⁵⁴ Department of Physics and Astronomy, University of the Western Cape, Bellville, Cape Town, 7535, South Africa
¹⁵⁵ DAMTP, Centre for Mathematical Sciences, Wilberforce Road, Cambridge CB3 0WA, UK
¹⁵⁶ Kavli Institute for Cosmology Cambridge, Madingley Road, Cambridge CB3 0HA, UK
¹⁵⁷ Department of Astrophysics, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland
¹⁵⁸ Department of Physics, Centre for Extragalactic Astronomy, Durham University, South Road, Durham DH1 3LE, UK
¹⁵⁹ IRFU, CEA, Université Paris-Saclay, 91191 Gif-sur-Yvette Cedex, France
¹⁶⁰ Oskar Klein Centre for Cosmoparticle Physics, Department of Physics, Stockholm University, Stockholm SE-106 91, Sweden
¹⁶¹ Astrophysics Group, Blackett Laboratory, Imperial College London, London SW7 2AZ, UK
¹⁶² Univ. Grenoble Alpes, CNRS, Grenoble INP, LPSC-IN2P3, 53, Avenue des Martyrs, 38000 Grenoble, France
¹⁶³ INAF-Osservatorio Astrofisico di Arcetri, Largo E. Fermi 5, 50125 Firenze, Italy
¹⁶⁴ Dipartimento di Fisica, Sapienza Università di Roma, Piazzale Aldo Moro 2, 00185 Roma, Italy
¹⁶⁵ Centro de Astrofísica da Universidade do Porto, Rua das Estrelas, 4150-762 Porto, Portugal
¹⁶⁶ HE Space for European Space Agency (ESA), Camino bajo del Castillo, s/n, Urbanizacion Villafranca del Castillo, Villanueva de la Cañada, 28692 Madrid, Spain
¹⁶⁷ Dipartimento di Fisica – Sezione di Astronomia, Università di Trieste, Via Tiepolo 11, 34131 Trieste, Italy
¹⁶⁸ Department of Astrophysical Sciences, Peyton Hall, Princeton University, Princeton, NJ 08544, USA
¹⁶⁹ Theoretical astrophysics, Department of Physics and Astronomy, Uppsala University, Box 516, 751 37 Uppsala, Sweden
¹⁷⁰ Mathematical Institute, University of Leiden, Einsteinweg 55, 2333 CA Leiden, The Netherlands
¹⁷¹ Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK
¹⁷² Space physics and astronomy research unit, University of Oulu, Pentti Kaiteran katu 1, FI-90014 Oulu, Finland
¹⁷³ Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, 59000 Lille, France

^★ Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Received: 18 March 2025
Accepted: 23 May 2025

Abstract

Modern astronomical surveys, such as the Euclid mission, produce high-dimensional, multi-modal datasets that include imaging and spectroscopic information for millions of galaxies. These data serve as an ideal benchmark for large, pre-trained multi-modal models, which can leverage vast amounts of unlabelled data. In this work, we present the first exploration of Euclid data with AstroPT, an autoregressive multi-modal foundation model trained on approximately 300000 optical and infrared Euclid images and spectral energy distributions (SEDs) from the first Euclid Quick Data Release. We compare self-supervised pre-training with baseline fully supervised training across several tasks: galaxy morphology classification; redshift estimation; similarity searches; and outlier detection. Our results show that: (a) AstroPT embeddings are highly informative, correlating with morphology and effectively isolating outliers; (b) including infrared data helps to isolate stars, but degrades the identification of edge-on galaxies, which are better captured by optical images; (c) simple fine-tuning of these embeddings for photometric redshift and stellar mass estimation outperforms a fully supervised approach, even when using only 1% of the training labels; and (d) incorporating SED data into AstroPT via a straightforward multi-modal token-chaining method improves photo-z predictions, and allow us to identify potentially more interesting anomalies (such as ringed or interacting galaxies) compared to a model pre-trained solely on imaging data.