© ESO 2018

1 Introduction

Next-generation experiments in Cosmology face the formidable challenge of understanding dark matter (DM) and dark energy (DE), two major components seemingly dominating the Universe content and evolution.

To improve our understanding of the Universe evolution history, the investigation of the distribution of galaxies over large volumes of the Universe at different cosmic times now constitutes a key requirement for future observational programs such as Euclid (Laureijs et al. 2011), WFIRST (Green et al. 2012), and LSST (Ivezic et al. 2008) that will exploit cosmological probes such as Weak Lensing (WL) and Galaxy Clustering (GC: baryon acoustic oscillations – BAO, redshift space distortions – RSD) to define the role of the dark components (Albrecht et al. 2006).

In GC, the detection of the BAOs at the sound horizon scale (r_s ≈ 105 h⁻¹ Mpc) is used to investigate the role of DE in the evolution of the expansion through measurements of the Hubble parameter H(z) and the comoving angular distances D_A(z) (Beutler et al. 2011), while the detection of the distorsions in the redshift space is used to probe the structures’ growth and DE models by measuring the parameter combination g_θ = f(z)σ₈(z), where f(z) and σ₈ refer to the growth rate and the rms amplitude (in a sphere of radius 8 h⁻¹ Mpc) of the density fluctuations (Beutler et al. 2012), respectively. The WL is used to map the matter distribution (dark + visible) in the Universe and constrain the expansion history through precise measurements of shapes and distances of lensed galaxies (Huterer 2002; Linder & Jenkins 2003).

In Cosmology, the redshift z is a fundamental quantity, which links distances and cosmic time through the use of a cosmological model. Accurate redshift measurements are at the core of all modern experiments aiming at precision cosmology for a better understanding of the Universe content, focused on the dominant DM and DE components, as the cosmological probes GC and WL that require precise redshift measurements to build robust statistical models to constrain the DE equation-of-state and investigate the content of the dark Universe (Abdalla et al. 2008; Wang et al. 2010). In particular, 3D galaxy distribution maps from GC measurements entail precise measurements of spectroscopic redshifts, while cosmic shear measurements in WL require, along with high-quality imaging and photometry, the selection of sources using redshift measurements for two reasons: First, the galaxies in front of the lens are not affected by the gravitational lensing but they dilute the signal of the galaxy source in the background, and second, the galaxies at the same redshift as the lens contribute to the intrinsic alignment that disrupts the WL measurements.

As part of the future large-scale experiments in Cosmology designed to address the DE and DM origin, the Euclid mission is a M-Class ESA mission from the ESA Cosmic Vision program that aims to probe the expansion and the LSS growth histories in the Universe. Through the combination of cosmological probes (BAO, RSD, WL, clusters of galaxies, supernovae – SNe), Euclid will achieve an unprecedented level of accuracy and control of systematic effects to derive precise measurements of the Hubble parameter H(z), the linear growth rate of structures γ, the DE equation-of-state parameters (ω_p, ω_a), the non-Gaussianity amplitude f_NL and the rms fluctuation of the matter over-density σ₈, among other cosmological parameters (Laureijs et al. 2011).

By covering a large fraction of the sky (Wide: ~15 000 deg², Deep: a total of 40 deg²), the mission will perform a photometric survey in the visible and three near-infrared bands to measure the weak gravitational lensing by imaging approximately 1.5 billion galaxies with a photometric redshift accuracy of σ_z ∕(1 + z) ≤ 0.05, in addition to a spectroscopic slitless survey of approximately 25 million galaxies with a redshift accuracy of σ_z ∕(1 + z) ≤ 0.001 in order to derive precise measurements of the galaxy power spectrum (Laureijs et al. 2011). The wide-field Euclid survey will be particularly challenging because of the large-size sample of faint distant galaxies, for which the spectroscopic redshifts need to be automatically measured, and their corresponding reliability evaluated.

For large-scale surveys such as Euclid, the sheer amount of data requires the development of robust and fully automated data-processing pipelines to analyze the data, extract useful information (e.g., redshift) and ensure that all requirements are met.

Distinctapproaches to estimate redshifts have been used in a broad range of galaxy surveys. Photometric redshifts z_phot are estimated using spectral energy distribution (SED) template fitting (e.g., Hyper-z Bolzonella et al. 2000; Le Phare Ilbert et al. 2006), classification with neural networks to produce a mapping between photometricobservables and reference data (e.g., ANNz, Collister & Lahav 2004), or Bayesian inference to compute a posterior z_phot PDF with prior information from integrated flux in filters, colour or magnitude: BPZ (Benitez 1999), ZEBRA (Feldmann et al. 2006), EAZY (Brammer et al. 2008). On the other hand, spectroscopic redshifts z_spec are estimated from the direct application of cross-correlation or chi-square-fitting methods between the observed data and a reference set of spectroscopic templates (Tonry & Davis 1979; Simkin 1974; Schuecker 1993; Machado et al. 2013), or using spectral feature detection (emission/absorption lines and continuum features including spectral discontinuities in the UV-visible domain such as the Lyman break or the Balmer and D4000A breaks) that can be very powerful (Schuecker 1993). Some codes (EZ, Garilli et al. 2010) combine spectral lines detection with cross-correlation or chi-square fitting to inject prior knowledge about more plausible redshift solutions.

Despite their overall performances in redshift estimation, most algorithms in use today still suffer from numerous modeling and computational deficiencies, as the major recurrent issues with the z_spec estimation algorithms remain the strong correlation between reliable spectral feature detection and the quality of the observed spectrum, the difficulty to define a representative set of reference templates, and the use of a pre-generated redshift grid Θ_z that might be beneficial for rapid and parallel processing but could induce a “bias” regarding the redshift space to probe.

In galaxy surveys, a key issue often overlooked is the necessary evaluation of the quality of a redshift measurement because spectroscopic redshift measurement methods may be affected by a number of known or unknown observational biases that may produce some errors in the output redshift, ranging all the way to a catastrophic measurement far from the real galaxy redshift. Further, despite the general trend that consists in linking the reliability of a redshift measurement to the S/N of detected spectral features, the noise in the data usually presents a strongly non-linear dependency on the flux spectrum for various reasons (e.g., the wavelength-dependency of the background flux), which makes the definition of a precise redshift reliability criterion even more difficult.

A number of previous faint galaxy surveys have adopted redshift reliability assessments, either by using empirical thresholds applied to a single metric operator (Baldry et al. 2014; Cool et al. 2013), or by combining independent reliability assessments performed by more than two experienced astronomers in order to smooth-out the observer bias of each individual and produce a remarkably repetitive reliability assessment (Le Fèvre et al. 2013, 2015; Garilli et al. 2014; Guzzo et al. 2014). All methods imply subjective information, either by selecting “adequate” thresholds from a constructed sample or by involving a human operator within the (visual) verification process that becomes largely unfeasible for samples over 10⁵ galaxies. For massive spectroscopic surveys such as Euclid or WFIRST, there is a critical need for a fully automated reliability flag definition that will adapt to the observed data and display a greater use of all available information.

In this paper, we propose to exploit a Bayesian framework for the spectroscopic redshift estimation to incorporate all sources of information and uncertainties of the estimation process (prior, data-model hypothesis), and produce a full z_spec posterior PDF, that will be the starting point of our automated reliability flag definition.

To test the proposed methodology of assessing the redshift reliability, we use a new redshift estimation software called algorithms for massive automatic Z evaluation and determination (AMAZED) developed as part of the Processing function (PF-SPE) in charge of the 1D spectroscopic data-processing pipeline of the Euclid space mission.

The paper is organized as follows. After introducing the subject, we present the data used in this study in Sect. 2, and in Sect. 3 we describe the Bayesian formalism of the spectroscopic redshift estimation. Section 4 is focused on the proposed automated reliability assessment method, where we first describe the principle, then present preliminary results of supervised and unsupervised classification techniques using the public database of the VIMOS VLT Deep Survey (VVDS, Le Fèvre et al. 2013). In Sect. 5, we present our results of redshift reliability predictions using preliminary simulations of Euclid spectra covering a wavelength range [1.25−1.85] μm, and we finally conclude in Sect. 6.

2 Reference data

To test the proposed method of assessing a redshift reliability, we use public data from the VIMOS VLT Deep Survey¹ (VVDS) in this study. The large VIMOS VLT Deep Survey (Le Fèvre et al. 2013) is a combination of three i-band magnitude limited surveys: Wide (17.5 ≤ i_AB ≤ 22.5; 8.6 deg²), Deep (17.5≤ i_AB ≤ 24; 0.6 deg²) and Ultra-Deep (23 ≤ i_AB ≤ 24.75; 512 arcmin²), that produced a total of 35 526 spectroscopic galaxy redshifts between 0 and 6.7 (22 434 in Wide, 12 051 in Deep and 1041 in UDeep) with a spectral resolution (R ≃ 230, dispersion 7.14A) approaching that of the upcoming Euclid mission (R ≥ 380 for a 0.5′′ object, dispersion 13.4A) as illustrated in Fig. 1.

The VIMOS Interactive Pipeline and Graphical Interface (VIPGI) data-processing software included background subtraction, decontamination, filtering and extraction of 1D spectra from 2D spectral images using sophisticated packages (Scodeggio et al. 2005). The VIMOS 1D spectroscopic data was processed using the EZ software (Garilli et al. 2010) to compute spectroscopic redshift measurements and reliability flags by combining reliability assessments (visual checks) of at least two experienced astronomers (Le Fèvre et al. 2013). The VVDS project provides a reference sample with a range of redshifts and reliability flags well-suited for testing our methods in a broad parameter space.

To evaluate our automated redshift reliability assessment method (Sect. 4), we use the VVDS data in two stages. First we exploit the existing redshift reliability flags of the VVDS data as a reference to assess the performances of supervised classification algorithms in predicting a similar redshift reliability label. Then, after partitioning the VVDS data into distinct clusters of redshift reliability flags using unsupervised classification, we compare these results with the original VVDS redshift flags to evaluate the performances of the proposed methodology and unveil possible discrepancies.

Fig. 1 VVDS-Deep galaxy spectra with identifiable spectral features at known redshifts: emission/absorption line, D4000A break.

3 Spectroscopic redshift estimation

3.1 Description

To derive a redshift, the widely used template-based algorithms rely on the hypothesis that “there exists a reference template spectrum that is a true (and sufficient) representation of the observed data”, implying that the observed spectrum can be described by at least one spectroscopic template of the reference library.

Using a set of rest-frame templates and a fixed grid of redshift candidates in Θ_z , for each pair (redshift z, template M_t) we compute the Least-Square metric: $${\chi }^2(z,t)={\displaystyle \sum _{i\in \Lambda }{\sigma }_i{}^{-2}}{(d_i-a{t}_{i,z})}^2,z\in {\Theta }_z\text{nn vspace}-0.2cm,$$(1)

or the cross-correlation: $$xc(z,t)=\frac{1}{{\sigma }_d{\sigma }_{t,z}}{\displaystyle \sum _{i\in \Lambda }(d_i-{\mu }_d)(t_{i,z}-{\mu }_{t,z})}{\sigma }_i{}^{-2},z\in {\Theta }_z,$$(2)

where d_i and σ_i refer respectively to the observed flux and noise spectra at pixel i, t_i,z is the redshifted template interpolated at pixel i, and (μ_t,z; σ_t,z ) and (μ_d ; σ_d ) are the mean and standard-deviation of the redshifted template and the observed spectrum respectively. The wavelength range in use Λ contains n datapoints, Θ_z refers to the redshift space to probe, and a is a scale factor referring to the amplitude of the redshifted template that is usually computed at each trial from (weighted) least-square estimation.

The estimated z_spec results from a joint-estimation of the pair (z, M_t) and is performed by optimizing a chosen metric: maximization of the cross-correlation function or minimization of the chi-square operator.

In general, the accuracy of the template-based methods is tied to the representativeness and wavelength coverage of the spectroscopic templates M_t in use.

3.2 Bayesian inference

Assuming a linear and Gaussian data model with i.i.d. (independent and identically distributed) residuals ${\left\{N_i\right\}}_{i\in \Lambda }$, the probability of observing the spectrum ${\left\{D_i\right\}}_{i\in \Lambda }$, at a redshift z given a template model M_t and any additional information I is described by the likelihood function $\mathcal{L}(z,M_t)$ (cf. Appendix A): $$\text{begin}align*\mathcal{L}(z,M_t)\text{ \&}=p(D|z,M_t,I)={\displaystyle \prod _{i\in \Lambda }p(N_i|z,M_t,I)}\text{ notagnn \&}={\displaystyle \prod _{1\le i\le n}{(\sqrt{2\pi }{\sigma }_i)}^{-1}}{\exp }^{-\frac{1}{2}{\chi }^2(z,t)},{\mathcal{l}}_{(z,M_t)}\text{ \&}=\log (\mathcal{L}(z,M_t))\text{ notagnn \&}=-\frac{1}{2}{\chi }^2(z,t)-\frac{N}{2}\log (2\pi )-{\displaystyle \sum _{i\in \Lambda }\log ({\sigma }_i)}.\text{end}align*$$(3) (4)

Via the Bayes rule, the joint posterior distribution is: $$\text{begin}align*\mathcal{L}(z,M_t)\text{ \&}=p(D|z,M_t,I)={\displaystyle \prod _{i\in \Lambda }p(N_i|z,M_t,I)}\text{ notagnn \&}={\displaystyle \prod _{1\le i\le n}{(\sqrt{2\pi }{\sigma }_i)}^{-1}}{\exp }^{-\frac{1}{2}{\chi }^2(z,t)},{\mathcal{l}}_{(z,M_t)}\text{ \&}=\log (\mathcal{L}(z,M_t))\text{ notagnn \&}=-\frac{1}{2}{\chi }^2(z,t)-\frac{N}{2}\log (2\pi )-{\displaystyle \sum _{i\in \Lambda }\log ({\sigma }_i)}.\text{end}align*$$(5) (6)

where π(z, t) is the joint-prior distribution of the pair (z, M_t).

The 1D posterior distribution is obtained by marginalizing over M_t : $$\text{begin}align*\mathcal{L}(z,M_t)\text{ \&}=p(D|z,M_t,I)={\displaystyle \prod _{i\in \Lambda }p(N_i|z,M_t,I)}\text{ notagnn \&}={\displaystyle \prod _{1\le i\le n}{(\sqrt{2\pi }{\sigma }_i)}^{-1}}{\exp }^{-\frac{1}{2}{\chi }^2(z,t)},{\mathcal{l}}_{(z,M_t)}\text{ \&}=\log (\mathcal{L}(z,M_t))\text{ notagnn \&}=-\frac{1}{2}{\chi }^2(z,t)-\frac{N}{2}\log (2\pi )-{\displaystyle \sum _{i\in \Lambda }\log ({\sigma }_i)}.\text{end}align*$$(7)

The “best” redshift ${\widehat{z}}_{spec}$ is the MAP (Maximum-A-Posteriori) estimate: $$z_{\text{MAP}}={\text{argmax}}_{z}p(z|D,I).$$(8)

This Bayesian formalism was not clearly stated for the spectroscopic redshift estimation. As for now, a posterior z_spec PDF can be computed and prior information, if available, can easily be integrated.

Furthermore if the hypothesis of the datamodel is readjusted, the equations can be rapidly and accurately revised in the likelihood expression (cf. Appendix A).

The template library used in this study includes a set of 9 continuum spectra of spiral, elliptical, starburst, and bulge galaxies, supplemented with 12 templates displaying different shapes and level for the continuum and the emission lines that were built by the VVDS team to take into account the diversity of galaxy spectra observed during the survey.

The spectroscopic templates that had only optical data were extended in the UV down to 912A by exploiting the closest templates with UV data, and below 912A by using nul flux spectra. In the infrared, a blackbody continuum was used to extrapolate the templates up to 20 000A.

This large wavelength coverage ensures that the intersection between the observed spectra and the templates is verified at each redshift trial.

3.3 Numerical computation

In the Bayesian inference, if our state of knowledge about a certain quantity θ is vague, a non-informative prior, such as the flat prior, is usually computed. $$\text{begin}align*\mathcal{L}(z,M_t)\text{ \&}=p(D|z,M_t,I)={\displaystyle \prod _{i\in \Lambda }p(N_i|z,M_t,I)}\text{ notagnn \&}={\displaystyle \prod _{1\le i\le n}{(\sqrt{2\pi }{\sigma }_i)}^{-1}}{\exp }^{-\frac{1}{2}{\chi }^2(z,t)},{\mathcal{l}}_{(z,M_t)}\text{ \&}=\log (\mathcal{L}(z,M_t))\text{ notagnn \&}=-\frac{1}{2}{\chi }^2(z,t)-\frac{N}{2}\log (2\pi )-{\displaystyle \sum _{i\in \Lambda }\log ({\sigma }_i)}.\text{end}align*$$(9)

Using a flat prior for redshift estimation implies that all redshifts and all templates are viewed as equiprobable solutions. The estimation algorithm will explore the full template library and the entire redshift grid and compute a (marginalized) posterior redshift PDF as displayed in Fig. 2.

If extra information about the pair (z, M_t) is available, the joint prior will be more informative as it will display a refined structure in the (z, M_t) space. For example, to estimate photometric redshifts, integrated flux in filters, colour, or magnitude can be usedas priors to efficiently probe the redshift space. In Benitez (1999), the joint-prior p(z, T|m₀) provides additional information about the most eligible spectral objects, T, with a magnitude, m₀ , that could be observed at certain redshifts, z. However, for spectroscopic redshift estimation, there is no clear definition of a (data-independent) prior, a choice justified by the fact that spectroscopic data is more informative than photometry.

Fig. 2 Posterior redshift PDF of two VVDS-Deep spectra. Top: a unimodal zPDF characterizes a very reliable redshift measurement (a single peak at z=0.879). Bottom: a multimodal zPDF refers to multiple redshift solutions (multiple peaks) possibly with similar probabilities associated to a diminished confidence level of the MAP estimate at z = 1.159. The quantity ${\delta }_{{\Theta }_z}$ refers to thefixed step of the redshift grid used to compute the zPDFs.

4 Reliability assessment

As the size of massive surveys in astronomy continues to expand, assessing redshifts’ reliability becomes increasingly challenging. The need for fully automated reliability assessment methods is now part of the requirements for future surveys, and is justified by the fact that automation provides predictable and consistent performances while the behavior of a human operator remains unpredictable and often inconsistent and therefore can require several independent observers to smooth out personal biases.

Moreover, the need for automation comes from the orders-of-magnitude increase in the total number of spectra that need to be processed. Visual examination of all spectra in a survey (2dF, DEEP2, VVDS, VIPERS, zCOSMOS, VUDS, PRIMUS, etc.) is extremely difficult for samples containing 10⁵ objects or more, and will be completely impossible for next-generation spectroscopy surveys with more than 50 × 10⁶ objects.

In general, existing approaches to automate the reliability assessment as well as the associated quality control in most engineering applications, such as the intrusion detection systems (IDS) that aim to evaluate the traffic quality by identifying any malicious activity or policy violation within a network, include:

• 1.

  Anomaly detection systems (ADS), where a component is labeled as an outlier if it deviates from an expected behavior using a set of thresholds or reference data (Chandola et al. 2009; Patcha & Park 2007). The ADS usually proceed by monitoring the system activity and detecting any sort of violation based on specific criteria or invariable standards.

• 2.

  Supervised classification that exploits prior knowledge of a referenced training set to predict a label (Shahid et al. 2014).

Both methods deliver great performances in general, but still have some limitations: irrelevant thresholds to new data for the ADS, and poor representativity of the training set in classification, and so on.

To automate the redshift reliability assessment, reproducing the ADS reasoning scheme by setting empirical thresholds might not be the best option when dealing with massive surveys. However, the use of machine learning (ML) techniques can still be a viable option but first requires the search for a valid model and a coherent set of entries.

In this work, the method to automate the redshift reliability flag definition stems from an attempt to address questions about the meaning of a “reliable” redshift:

• 1.

  What guides an experienced astronomer to declare an estimated redshift as a plausible solution; apart from visual inspection of the data and its fitted template?

• 2.

  Is there some disregarded information within the z-estimation process that we can further exploit?

• 3.

  How cana system “perceive” the same information as a human does?

Spectroscopic redshift measurements are obtained from χ² minimization or maximization of the posterior probability p(z|D, I) in Bayesian inference (cf. Sect. 3), and usually no further analysis of the computed functions is conducted afterwards. When computing the posterior redshift PDF, broadly two types of probability density function can be observed (cf. Fig. 2): a unimodal PDF versus a multimodal distribution. In both cases, a pipeline will provide a redshift estimation z_MAP but the estimated redshifts from these two different types of PDFs definitely do not show the same level of reliability. In fact, the multimodal PDF refers to numerous redshift candidates possibly with similar probabilities, while a strong unimodal PDF with a prominent peak and low dispersion depicts a more “reliable” redshift estimation of the data.

We exploit such characteristics of the posterior PDF to build a discretized descriptor space that will be the entry point for ML techniques to predict a reliability label. Our approach aims to build the “experience” of an automated system in order to assess the quality of a redshift measurement from the zPDF.

4.1 Description

In machine learning, the typical entries of the model are a response vector Y and a feature matrix X: $$X=\left(\begin{array}{c}{x}_1\\ ⋮\\ x_M\end{array}\right)=\text{nn begin}blockarraycccc\text{nn \&}{d}_1\text{ \& \&}{d}_P\text{nn begin}blockc(ccc)s_1\text{\&}{x}_{11}\text{n\&}\cdots \text{\&}{x}_{1P}\text{nn nnnn \&}⋮\text{\&}\ddots \text{\&}⋮s_M\text{\&}{x}_{M1}\text{\&}\cdots \text{\&}{x}_{MP}\text{nn end}block\text{nn end}blockarray;\overrightarrow{Y}=\left(\begin{array}{c}{y}_1\\ ⋮\\ y_M\end{array}\right)\text{nn vspace}-2mm,$$(10)

where x_j = (x_j,1…x_j,P) is the P-dimensional feature vector of the jth observational data ${(s_j)}_{j\in \{1,\dots M\}}$, and y_j is its response variable. If the response vector Y of the model is unknown, the prediction of a label y_j using only the distribution of X in the feature space refers to clustering (unsupervised classification). Otherwise, we talk about supervised classification whose goal is to define a mapping between the observable entries X and their associated response variables Y through a dual training/test scheme.

In ML, the design of the entry model is decisive. What could be the optimal selection of informative and independent features to accurately describe the zPDF? Can a single operator, such as the integral under the redshift solution z_MAP or the difference in probability between the first two peaks (modes), be a unique and sufficient descriptor? No definite answers can be given, since this approach of “quantifying the spectroscopic redshift reliability” from the zPDF is new. Each set of selected features will define a different descriptor space that a classifier could separate differently.

In this study, our selected ML entries are redshift reliability flags (Y) and descriptors of the zPDFs (X), where the feature vector x_j associated to the observation s_j = {D} consists of alist of eight tailored descriptors of the zPDF:

• The quantity $P(z_{\text{MAP}}|D,I)\approx p(z_{\text{MAP}}|D,I)\times {\delta }_{{\Theta }_z}$, where ${\delta }_{{\Theta }_z}$ is the fixed step of the redshift grid.

• The number of significant modes in the PDF. The “significance” of a mode is determined by partitioning the set of detected peaks of the PDF into two categories (strong/weak) based on their prominence and height in order to avoid including the extremely-low density peaks (10⁻¹⁰⁰ usually) that result from the conversion of logPDFs into a linear scale.

• The difference in probability of the first two best redshift solutions (z_MAP, z₂): P(z_MAP|D, I) − P(z₂|D, I).

• The dispersion $\sigma ={[{\displaystyle \int {(z-\overline{z})}^2}p(z)\text{d}z]}^{1/2}$, with $\overline{z}={\displaystyle \int z}p(z)\text{d}z$.

• The cumulative probability in the region $R_2^{*}$: $[z_{\text{MAP}}\pm \delta ]$ where the parameter δ is chosen equal to 0.001.

• The characteristics of the CR^*(restricted version of the credibility region with 95% in probability): number of z candidates, width Δz, cumulative probability. In Bayesian Inference, the CR is analogous to the frequentist CI (confidence interval). For a 100(1 − α)% level of credibility, the CR is defined as: ∫ _CRp(z|D, I)dz = 1 − α. The restricted CR^* used sets (optional) maximal bounds to the search region around z_MAP to accelerate the operation.

Displays of distinct zPDFs are presented in Figs. 3–6, where the descriptors listed above highlight interesting features of the zPDFs. For example, it is possible to obtain a similar dispersion for two zPDFs but a different number of significant redshift modes (Fig. 3), or the other way around: multimodal zPDFs with a comparable number of redshift modes with different amplitudes, and a different dispersion (cf. Fig. 4) or difference in probability between the first two best redshift solutions (cf. Fig. 5). Also, unimodal zPDFs can vary as they can display wider or narrower restricted CR (cf. Fig. 6) or different values of the dispersion σ.

Using the eight listed key descriptors, we estimate that the main features of the zPDF can be inferred. This design is not immutable. Supplementing the feature matrix with additional information about the observed spectra, s, or designing a different feature selection can also be explored.

Fig. 3 Display of two zPDFs with multiples modes and a similar dispersion.

Fig. 4 Display of two zPDFs with multiples modes and a different dispersion.

Fig. 5 Display of two zPDFs with multiples modes and different characteristics of the two best z solutions.

Fig. 6 Display of two zPDFs with a single mode and a different width of CR * .

4.2 Classification

4.2.1 Model

The ML entries in this study are obtained from a collection of zPDFs computed from M spectra of the VVDS to which a reliability label ${(y_i)}_{i\in \{1,\dots M\}}$ is known to belong to one of the flags (Le Fèvre et al. 2005, 2013):

• Flag 1, “Unreliable redshift”.

• Flag 2, “Reliable redshift”.

• Flag 9, “Reliable redshift, detection of a single emission line”.

• Flag 3, “Very reliable redshift with strong spectral features”.

• Flag 4, “Very reliable redshift with obvious spectral features”.

The redshift reliability flags in the VVDS are determined by confronting independent redshift measurements performed by several observers onthe same spectra.

By comparing the redshift measurements with internal duplicated observations or with published redshifts from different surveys, the VVDS spectroscopic redshift flags have been empirically paired with a probability for “a redshift to be correct”: the VVDS redshift reliability flags {1, 2, 9, 3, 4} are associated with probabilities of [50–75]%, [75–85]%, ~80%, [95–100]%, and 100% , respectively, that the measured redshifts are correct.

Using supervised classification, the objective is to predict similar redshift reliability flags for new unlabeled data. However, since the reproducibility of the VVDS redshift reliability flags is difficult because of their subjective definition and the confusion between “quality of a redshift” and “specific information about the data”, we first decided to regroup the VVDS flags, as following:

• “Class 0”, consisting of the “VVDS flags 1” to depict the uncertain redshifts.

• “Class +1”, consisting of the “VVDS flags 2–9” to depict the reliable redshifts.

• “Class +2”, consisting of the “VVDS flags 3–4” to depict the very reliable redshifts.

A three-class classification problem is then set. For multi-class problems, the error-correcting-output-codes (ECOC), as introduced in Dietterich & Bakiri (1995), are adapted for several learners, such as support vector machines (SVM), Tree templates, and Ensemble classifiers. A description of the ECOC is provided in Appendix B.

4.2.2 Preliminary tests

Classification tests are conducted using a VVDS subset of 24519 spectra with a constraint on the redshift accuracy |z_MAP − z_ref|∕(1 + z_ref) ≤ 10⁻³ for the VVDS flags {2, 9, 3, 4}. Our main objective is to build a descriptor space from a diverse set of zPDFs and evaluate the ability of the system to predict a redshift reliability label.

The dataset is decomposed into a “Training set” and a “Test set” (cf. Tables 1–3).

Different classifiers are tested in this study to carry out a careful analysis and avoid blindly trusting the results in cases of overfitting. We assess that different techniques should provide a different but not very disparate level of performance. Three classifiers are selected: the SVMs with linear and Gaussian kernels, an ensemble of bagging trees (referred to simply as TreeBagger) and a GentleBoost ensemble of decision trees. A general description of the classifiers and the multi-class measures is provided in Appendices C and D.

To evaluate the performance of a classifier, two tests are conducted:

• Test 1: resubstitution.

• Test 2: test prediction.

In the resubstitution, the “Training set” is reused as the “Test set” during the prediction phase. Extremely low prediction errors are expected ( ≲1% classification error rate): if a bijective relation exists between the observables X_train and the response vector Y_train, the generated mapping from the training phase is supposedly accurate. The predicted labels Y_pred in resubstitution tests are therefore expected to resemble the true labels Y_train with high accuracy, otherwise a clear mismatch between the features matrix X and the response vector Y of the ML model is reported. In such a case, the predictions of the second test (“Test prediction”) would be baseless, since the mapping produced from the training phase is truly unusable. The overall performances reportedin Tables F.9 and F.10 in addition to the confusion matrices (cf. Tables F.1 to F.8) representing the fraction of the predicted labels versus the true classes in Y_test, support thisconclusion. Most classifiers seem unable to predict the true labels in resubstitution: non-zero off-diagonal elements in the matricesand a high error-rate, implying that a correct mapping between the feature matrix and the existing VVDS redshift reliability flags cannot be produced.

We would like to point out the singular case of the TreeBagger that seems to generate a good mapping in resubstitution (error rate 0.08% on average) in comparison with the SVMs that are commonly-known as robust classifiers (error rate >10% in average). It seems reasonable to consider that the observed dissimilarity between the different classifiers in resubstitution is due to the sensitivity of the bagging trees to several parameters as the number of learners or the trees depth that can coerce the training into focusing on irregular patterns and establish an erroneous mapping). As anticipated from the resubstitution results (high error rate), we also find that the test predictions present a significant error rate ( ~ 40% on average).To summarize, these first results of supervised classification show that trying to match the subjective VVDS flags with descriptors of the zPDF gives poor results.

The entries and hypotheses for ML have to be reexamined.

4.3 Clustering and fuzzy classification

From the previous results, doubts can be raised regarding the engineered zPDF feature space derived from a collection of 24519 VVDS spectra. However the selected set of descriptors seems to be a viable description that portrays an existing but hidden structure of the feature space.

Clustering, known as unsupervised classification, is used in this Section to unveil the intricate structure and bring into light some properties of the data in the descriptor space.

4.3.1 Partitioning the descriptor space

In unsupervised classification, prior knowledge about class membership is unavailable. Partitioning the descriptor space into K manifolds is realized by applying separation rules only to the feature matrix X .

By representing the zPDFs feature matrix X in 3D (cf. Fig. 7), a simple bi-partitioning is introduced:

• Group 1: high dispersion and low P(z_MAP|D, I) referring to multimodal PDFs or platykurtic unimodal PDFs.

• Group 2: medium dispersion and high P(z_MAP|D, I) depicting strongly peaked unimodal PDFs.

In each category, we choose to reapply a bi-partitioning to decompose the data into a dichotomized pattern (cf. Fig. 8). This partitioning strategy, applied to the entire descriptor components and not only to the two descriptor components as in the displays, alongside with the number of clusters, the feature selection and the ML algorithms tested in this work as a novelty to automate the redshift reliability, are not immutable and can be readjusted according to the data in hand. Further evaluations will be conducted on these aspects of ML to develop a robust and precise automated assessment of redshift reliability.

Using the classic clustering algorithm FCM (Fuzzy C-Means) to minimize the intraclass variance (cf. Appendix E), the final groups identify distinct partitions in the feature space (cf. Figs. 9 to 11). In this study, the selection of the number of clusters is an empirical process based on the analysis of the intermediate partitions and testing different configurations. We assess that the final architecture is a viable solution amongst others.

• “Cluster C1”: highly dispersed PDFs with multiple equiprobable modes, P(z_MAP) ~ 0.028 ± 0.023.

• “Cluster C2”: less dispersed PDFs, with few modes and low probabilities P(z_MAP) ~ 0.087 ± 0.033.

• “Cluster C3”: low σ, intermediate probabilities P(z_MAP) ~ 0.166 ± 0.035.

• “Cluster C4”: unimodal PDFs with low dispersion, higher probabilities P(z_MAP) ~ 0.290 ± 0.059.

• “Cluster C5”: strong unimodal PDFs with extremely low dispersion, better probabilities P(z_MAP) ~ 0.618 ± 0.204.

The coordinates of the clusters’ centroids in the descriptor space are reported in Table 4. $$\{\begin{array}{ll}\text{Class }\left\{C_k\right\}\text{ centroid}\hfill & g_k=\frac{1}{M_k}{\displaystyle \sum _{y_j\in C_k}{x}_j}\hfill \\ \text{Class }\left\{C_k\right\}\text{ variance}\hfill & V_k=\frac{1}{M_k}{\displaystyle \sum _{y_j\in C_k}{(x_j-g_k)}^{\top }}(x_j-g_k),\hfill \end{array}$$(11)

where M_k is the number of elements in cluster C_k.

Tables 5 and 6 report the intraclass dispersion $\sqrt{W}$ and the interclass dispersion $\sqrt{B}$ that characterize the newly defined clusters: $$\{\begin{array}{ll}\text{Interclass variance}\hfill & B=\frac{1}{M}{\displaystyle \sum _k{M}_k}{(g_k-g)}^{\top }(g_k-g)\hfill \\ \text{Intraclass variance}\hfill & W=\frac{1}{M}{\displaystyle \sum _k{M}_k}{V}_k,\hfill \end{array}$$(12)

with the total variance: $$V=\frac{1}{N}{\displaystyle \sum _{j=1}^M{(x_j-g)}^{\top }}(x_j-g)=B+W,$$(13)

where g is the global centroid and M is the full number of elements in the descriptor space.

The variance tables show that the intraclass variance, W, is generally small in comparison to the interclass variance, B, except for two descriptors (the dispersion and the number of modes). This results from the fact that the cluster C1 allows wider variations for these two components. Since the class C1 refers by definition to multimodal zPDFs associated to very unreliable redshift measurement, the results remain coherent.

Given the possibility that the clustering results might be unreliable due to inherent computational limitations or an incorrect modeling of the descriptor space, the full content of each partition ${(C_k)}_{k\in \{1,2,3,4,5\}}$ is investigated. We find that, overall, the zPDFs within each class, C_k , verify the properties listed above. The newly defined partitions genuinely describe a homogeneous representation of the data in the feature space.

Fig. 7 3D representation of the feature matrix X. Broadly, two categories are noticeable. The first group refers to the zPDFs with high dispersion, large zMAP peak and low probability P(zMAP|D, I) that can be assimilated to multimodal PDFs or platykurtic unimodal PDFs. The second group characterizes the zPDFs with medium-to-low dispersion, narrow zMAP peak and high P(zMAP|D, I) that depict strongly peaked unimodal PDFs.

Fig. 8 Clustering the zPDFs features X in a dichotomized pattern. The clustering strategy exploits the classic FCM algorithm at each step to decompose the input data into two sub-classes using the entire set of descriptors. The final categories ${\left\{C_k\right\}}_{k\in \{1,2,3,4,5\}}$ are displayed in distinct colors.

Fig. 9 Representative zPDFs for each clusters. Display of representative zPDFs in each cluster obtained from clustering. The shift in the confidence level in the ${\left\{C_k\right\}}_{k\in \{1,2,3,4,5\}}$ clusters is apparent in the type of the zPDF: from multimodal zPDFs to unimodal zPDFs with narrower zMAP peaks, the confidence level ranges from “extremely unreliable redshift estimate” (C1) to “very certain redshift estimate” (C5).

Fig. 10 Clusters repartition in a selected 3D space. The five zReliability clusters described in Sect. 4.3.1 are associated to different types of redshift PDFs, where the two extreme categories C1 and C5 , respectively,describe highly dispersed multimodal zPDFs and peaked unimodal zPDFs.

Fig. 11 Distribution of the probability value P(zMAP|D, I) within the five partitions. The distribution of the probability values (component of the descriptor space) P(zMAP|D, I) show distinct properties of the clusters. The five zReliability clusters (cf. Sect. 4.3.1) are associated with different categories of redshift PDFs.

4.3.2 Cluster analysis

In this section, we compare the initial VVDS redshift reliability flags and the new clusters in order to point out peculiar cases of misclassifications: unexplained discrepancies between the manually attributed flags in the VVDS database and those resulting from the unsupervised classification (cf. Sect. 4.3.1).

Two examples of misclassification are reported in Figs. 12 and 13:

• 1.

  A misclassification of a “VVDS Flag 1: unreliable redshift estimation” as C5 (unimodal zPDF and very reliable ${\widehat{z}}_{\text{spec}}$) is presented in Fig. 12. The misclassification is due to the mismatch between the flux spectrum and its noise component, where the latter seems very inadequate when considering the good quality of the data. A problem regarding the generation of the 1D data (flux & noise components) from the 2D → 1D extraction can be noted.

• 2.

  A different type of misclassification illustrated in Fig. 13, where a “VVDS flag 9: secure redshift estimation with an identifiable strong EL” is identified as C1 (for very multimodal zPDFs and extremely unreliable ${\widehat{z}}_{\text{spec}}$). This discrepancy between the VVDS flag and the new label from clustering could be ascribed to an imprecise computation of the zPDF due to a lack of representative templates at the given redshift, or a biased evaluation of a human operator.

To evaluate the misclassification rate for the entire VVDS dataset used in this study, Tables 7 and 8 summarize the repartition of the initial VVDS flags {1; 2;9;3;4} within the predicted reliability clusters.

We find that:

• The green cells represent the “expected” behavior: the cluster C1 is mainlycomposed of the unreliable redshift “VVDS flags 1” ( ~86%), while themajority of the “VVDS flags 4” are in C4/C5 ( ~81%) and the “VVDS flags 3” are in C3/C4 ( ~68%).

• The graycells represent a “gray area”: the clustering provides homogeneous partitioning in comparison with the VVDS flags, as it properly incorporates the full information from the input data (cf. observed flux and its associated noise component).

We find that the “VVDS flags 2−9” in C4/C5 ( ~20% each) are associated with extremely bright objects with easily identifiable spectral features that make the estimated redshifts very secure.

On the other hand, the “VVDS flags 4” predicted in C3 ( ~15%) are associated to noisier spectra with scarce spectral features in comparison with the “VVDS flags 4” in C5. The redshift reliability level for these spectra is thereby diminished.

Similarly, the prediction of “VVDS flag 2” in C2 ( ~43%) is due to the degradation of data quality in comparison with the “VVDS flags 2” located in C3.

The main reason behind these discrepancies lies in having different observers conducting the redshift-quality checks, as each person has their own understanding of a “redshift reliability” depending on their experience and knowledge of objectively assessing whether a redshift is deemed a secure estimation or not.

• The red cells are associated with peculiar cases of “abnormal” zPDFs resulting from incorrect noise spectra and/or human misclassification. In particular, the 71 cases listed of “VVDS flag 1” in C5 result from a mismatch between the flux and noise components; the noise component seems extremely low considering the reduced data quality. Having very low noise components contributes to reinforce that the flux information depicts a real observation even when it is not the case. We obtain, finally, extremely peaked zPDFs that are predicted as C5.

For the 23 spectra of “VVDS flag 4” in C1, 13 cases are related to highly dispersed multimodal zPDFs where a confusion between the oxygen emission line [OII]_3726A and Ly α is reported: both emission lines are strong candidates which gives at least two significant modes detected in the zPDF. Also, the fact that the associated peaks are very distant in the redshift space results in a high dispersion value, σ, of the zPDF. The prediction in C1 is highly driven by these characteristics. We also report four cases within these 23 spectra that are associated to low S/N spectra: an important noise component annihilates the confidence in the flux vector and therefore produces highly multimodal zPDFs predicted in C1. For the remaining six cases of “VVDS flag 4” in C1, they result from an excessively-high noise component that produces very degenerate zPDFs, also predicted in C1.

The main result from the cluster analysis is that existing redshift reliability flags cannot be reproduced with a 100% accuracy due to their subjective definition, however a general trend can be retrieved as the majority of the VVDS initial redshift flags can be described by one or two of the redshift reliability clusters ${\left\{C_k\right\}}_{k\in \{1,2,3,4,5\}}$.

Fig. 12 Misclassification – case 1. A “VVDS flag 1: very unreliable redshift” is predicted by the classifier in the new category “C5” for very reliable redshifts. The spectrum displays very low noise components that reinforce the confidence in the measured flux pixels. However the extracted 1D spectrum appears distorted considering the initial 2D spectrum. The extraction 2D → 1D induced a bias in the estimation of the (falsely unimodal) zPDF.

Fig. 13 Misclassification – case 2. A “VVDS flag 9: reliable redshift, detection of a single emission line” is predicted by the classifier in the new category “C1” for very unreliable redshifts. The spectrum displays a strong noise component that annihilates the confidence in the measured flux pixels (especially the spectral emission line [OII] 3726 at 7365A). Several redshift solutions are declared as plausible solutions (a multimodal zPDF).

4.3.3 Re-using the clusters for redshift reliability label predictions

– Classification tests

The clustering results showed a great coherency between the automated definition of redshift reliability labels using the zPDF’s features matrix and our understanding of “a redshift reliability”.

The idea presented in this Section consists in re-using the new labels of the 24519 VVDS spectra as the response vector Y _train in supervised classification, to train a classifier to predict redshift reliability labels for new unlabeled data. For this purpose, classification tests are performed using the Training and Test sets in Tables 9 and 10. The resubstitution and test predictions are also used to verify once again the accuracy of the partitioning and objectively assess whether the FCM dichotomized strategy produced “random results” or a “a true description of the zPDFs” in the descriptor space.

Similar performances are observed for several classifiers in resubstitution, with extremely low off-diagonal elements in the confusion matrices and an average per-class error rate ≲ 1% (cf. Tables F.11 to F.14, and Table F.19) for all four classifiers, which is a clear contrast with the results in Sect. 4.2. By having low resubstitution errors, the mapping is deemed a reliable reproduction of the input data, and the prediction of X_test can be examined. We find in test predictions that the confusion matrices for several classifiers offer a good predictive power (average per-class error rate < 2%), with the Linear SVM scoring slightly lower results (cf. Tables F.15 to F.18, and Table F.20).

– Fuzzy approach In ML, two main approaches exist: “hard” partitioning where an object is said to belong to a unique class (binary membership), and “soft/fuzzy” partitioning where the membership of an object to a class is expressed in terms of a probability between 0 and 1 (Wahba 1998, 2002).

In the classification tests, “soft” partitioning is used to compute the posterior class prediction probability in order to evaluate the classifier predictive power. The class posterior probabilities $p(Label|\left\{C_1,C_2,C_3,C_4,C_5\right\};Descriptors)$ are obtained by minimizing the Kullback-Leibler divergence (Hastie & Tibshirani 1998).

In the test predictions on the evaluated VVDS dataset, we find that most class prediction probabilities fall between 0.7 and 1.0 (bright colors in Fig. 14). However, we estimate that it could be possible for new data to be assigned to a redshift reliability label with a lower probability, meaning that the classifier cannot project with certainty the unlabeled zPDF into the descriptor space as a result of an incorrect PDF (numerical limitations, degraded input spectra, etc.), or if it is located close to the margins of two or more clusters. For such cases, a new class of “Unidentified” objects has to be set apart from the labeled clusters ${\left\{C_k\right\}}_{k\in \{1,2,3,4,5\}}$. This particular point on the class prediction using soft partitioning is addressed further in the following section.

Fig. 14 Class posterior probabilities. The predictive power of several classifiers is displayed for each true class in Y test . Most prediction probabilities fall between ~[70–100]% (bright colors). For example: the Linear SVM correctly predicts ~92% (2060 elements) of the subset of “true C2” (around 2240 elements) in Y test (cf. Table F.17) with class prediction probabilities between 0.7 and 1 (bright colors).

5 Tests on mock simulations for the Euclid space mission

An end-to-end simulation pipeline is currently under development for Euclid using catalogs of realistic input sources with spectro-photometric information and an instrumental model for the spectrophotometer NISP designed to perform slitless spectroscopy and imaging photometry in the near-infrared (NIR) wavelength domain. For Euclid, observationsof the same field will be obtained from the combination of three or more different roll angles (referring to different orientations of the grisms) in order to alleviate the superposition of overlapping spectra due to the slitless mode.

Using the pixel simulator software TIPS (Zoubian et al. 2014), 1D spectra are obtained from 2D dispersed images after subtracting the sky background from the raw data and combining co-added image stamps of different roll angles. In these preliminary simulations for Euclid, a contamination model (zodiacal light, adjacent sources, etc.) is not included.

Table 11 reports the main characteristics of the simulated data of H α EL galaxies at redshifts in the range 0.95 ≤ z ≤ 1.40.

The Euclid simulations are not associated with a redshift reliability flag, and thereby are qualified as “unlabeled data” in this work. To test the performance of the redshift reliability assessment method, two sets of unlabeled spectra are used (S1, S2), with a total of 3169 spectra per set.

By varying the source size and the sky background level, the difference in data quality between the two datasets is noticeable: Fig. 15 displays sample spectra for each dataset.

Fig. 15 Simulated Euclid spectra. Left: simulated galaxy spectrum (id: 53678850) in the dataset S1 with an identifiable H α line at 12803A. Right: simulated galaxy spectrum (id: 56932048) in the dataset S2 with a H α emission line at 12908A.

5.1 Reliability class predictions

The redshift PDFs of the Euclid simulated datasets are computed using a constant prior in Θ_z (cf. Figs. 16 and 17), and projected into the mapping (cf. Sect. 4.3) using soft partitioning to predict redshift reliability labels. Class prediction results are reported in Tables 12 and 13.

The system computes predominantly multimodal zPDFs with high dispersion when the useful information cannot be retrieved from the data because of low S/N: the estimated redshifts are deemed unreliable, which explains the high percentage of S2 spectra in the clusters C1∕C2 ( ~52.3%). In contrast, for high S/N data, the system identifies the majority of redshifts as very reliable: high percentage of S1 spectra in the clusters C4∕C5 ( ~88.1%).

Moreover, the highlighted cells (in magenta) within the result tables indicate two particular cases we anticipated to be null fractions when considering the data quality: We denote on one hand few spectra in the dataset S1 (high S/N) that are associated with unreliable redshift measurements ( ~ 2% predicted in C1∕C2), and on the other hand a small fraction of spectra in S2 (low S/N) that is linked to very reliable redshifts ( ~ 9% predicted in C5). Such results can easily be understood by looking at the distribution in [ log (f H α ), J_AB] of the input spectra (cf. Fig. 18). We find that:

• Bright objects are mainly located in C5, while themajority of faint objects are predicted as C1/C2, in particular when the flux spectrum is embedded in a strong noise (S2). This distribution can be assimilated to a shift C1 → C5 according to the intrinsic properties of the observed object.

• The difference in absolute values (cf. Table 12) between the results in S1 and S2 is due to the increased noise level from the sky background that injects a higher uncertainty in the observed flux spectrum. The redshift reliability is decreased in S2 in comparison with less noisy data (S1). The repartition in absolute values seems to describe a shift C5 → C1 according to observational constraints (S/N).

Fig. 16 Computed zPDF for a galaxy spectrum in the dataset S1. The redshift probability density function is computed using a constant prior over Θz.

Fig. 17 Computed zPDF for a galaxy spectrum in the dataset S2. The redshift probability density function is computed using a constant prior over Θz.

Fig. 18 log(f H α ), JAB distribution of the reliability class predictions for unlabeled simulated galaxy spectra for Euclid. The number of faint objects predicted in C1/C2 increases when the noise component in the data is important (S1 → S2). The increased sky background injects a strong noise component of the spectra that annihilates the confidence in a measured redshift, resulting in multimodal zPDFs with high dispersion (C1/C2). In contrast, extremely bright objects with an identifiable Hα line are located in C4∕C5, because the redshift estimation is deemed very reliable when distinct spectral features are found.

5.2 Redshift error distribution

We further investigate the distribution of the redshift error ε_z = |z_MAP − z_ref|∕(1 + z_ref) within the predicted clusters (cf. Table 14).

We find that the majority of incorrect redshift estimations (ε_z > 10⁻³) are located in the clusters C1/C2 for “unreliable redshifts” since low S/N data are more likely to be associated with inaccurate redshift measurements.

For the two datasets, the fraction of spectra associated with low redshift error (ε_z ≤ 10⁻³) is ~100%, ~99%, ~95%, and <70% in C5/C4, C3, C2, and C1, respectively.

From this particular result, one approach would be to identify a possible correlation between the redshift reliability clusters and a specific range of redshift errors in order to define a probability for “a redshift to be correct” within the {C_k } clusters, in a similar way to that used for VVDS.

In this direction, the next step will be to conduct similar tests on a wide basis of Euclid simulated datasets (with a contamination model) to statistically constrain the correlation between redshift errors and the redshift reliability clusters.

5.3 Fuzzy approach for label prediction

As previously stated in Sect. 4.3.3, soft partitioning in ML provides extra information about the classifier predictive power that can be affected by several factors as possible outliers in the training set or numerical limitations associated to the zPDF computation.

In this study, we find that the majority of S1 and S2 spectra are associated with class probability predictions higher than 99%, as in the example of Table 15. However, peculiar cases, related to class predictions falling within the margins of two or more reliability clusters are detected, as in the example reported in Table 16 where the class posterior probabilities of the cluster C4 are quite close to the predicted class C3.

In this study, the predictions associated to lower-class probabilities are extremely few, with a confusion clearly stated between adjacent clusters (C1 with C2 or C4 with C5 for example).

A confusion entailing predicting a label C4/C5 as C1 (and vice-versa) could have been more problematic and can result from an erroneous computation of the zPDF or an incorrect spectroscopic data (flux and noise components). We estimate that soft partitioning can be used to unveil such peculiar cases and improve the clustering by identifying possible outliers in the descriptor space that can be assigned to the “Unidentified” class independently from the ${\left\{C_k\right\}}_{k\in \{1,\dots 5\}}$ clusters.

	multi-class classification ${(C_k)}_{k\in \{1\dots K\}}$
measures per-class	accuracy(Ck)	$\frac{T{P}_k+T{N}_k}{T{P}_k+F{N}_k+T{N}_k+F{P}_k}$
	precision(Ck)	$\frac{T{P}_k}{T{P}_k+F{P}_k}$
	sensitivity(Ck)	$\frac{T{P}_k}{T{P}_k+F{N}_k}$
	f-score(Ck)	$\frac{2*T{P}_k}{2*T{P}_k+F{N}_k+F{P}_k}$
	specificity(Ck)	$\frac{T{N}_k}{F{P}_k+T{N}_k}$
average per-class	accuracy	$\frac{1}{K}{\displaystyle \sum _{k}A}ccuracy(C_k)$
	error rate	$\frac{1}{K}{\displaystyle \sum _{k}E}rror(C_k)$
	precision	$\frac{1}{K}{\displaystyle \sum _{k}P}recision(C_k)$
	sensitivity	$\frac{1}{K}{\displaystyle \sum _{k}S}ensitivity(C_k)$
	f-score	$\frac{1}{K}{\displaystyle \sum _{k}F}score(C_k)$

5.4 Discussion

The results obtained using preliminary mock simulations for Euclid show that the new automated reliability redshift definition can be used to quantify the reliability level of spectroscopic redshift measurements. This method could be useful for cosmological studies that require accurate redshift measurements. By using 1D spectra of newly released Euclid simulations, upcoming studies will focus on the correlation between the distribution of redshift errors and the redshift reliability clusters to define the probability for “a redshift to be correct” in the ${\left\{C_k\right\}}_{k\in \{1,\dots 5\}}$ clusters in asimilar approach as in VVDS.

6 Summary and conclusions

By mapping the posterior PDF p(z|D, I) into a discretized feature space and exploiting ML algorithms, we are able to design a new automated method that correlates relevant characteristics of the posterior zPDF, such as the dispersion of the probability distribution and the number of significant modes, with a reliability assessment of the estimated redshift.

The proposed methodology consists of three steps:

• 1.

  Using a set of representative spectra, compute the redshift posterior PDFs p(z|D, I) and extract a set of features to build the descriptor matrix X.

• 2.

  Generate a reliable partitioning Y of the feature space using clustering techniques and prior knowledge, if available.

• 3.

  Use the partitioning to train a classifier that will predict a quality label for new unlabeled observations.

Using the zPDFs descriptors, we first tried to bypass the first two steps by exploiting existing reliability flags to train a classifier (supervised classification), but the results obtained (Sect. 4.2) justify the need for new homogeneous partitions [steps 1 and 2] of the feature space because the reproducibility of the existing quality flags cannot be achieved due to their subjective definition: the combination of several visual checks performed by different observers cannot derive homogeneous and objective criteria of redshift reliability for an automated system to learn from.

The results of unsupervised classification in Sect. 4.3 displayed great coherency in describing distinct categories of zPDFs: the multimodal zPDFs with equiprobable redshift solutions and high dispersion, versus the unimodal zPDFs with a narrower peak around the z_MAP solution, each depicting a different level of reliability for the measured redshift.

To predict a redshift reliability flag for unlabeled data (Sect. 5), our methodology consists in projecting the unlabeled zPDF [step 3] into the mapping generated from known zPDF descriptors X and their associated z reliability labels Y to predict the class membership.

A fuzzy approach can also be used to predict the class prediction probability and provide relevant information about the classifier performance and possible discrepancies in the input data.

To conclude, the proposed method to automate the redshift reliability assessment is simple and flexible; the only requirement being robust redshift estimation algorithms with representative templates and a good computational efficiency to produce accurate redshift PDFs. For the spectroscopic redshift estimation, the use of the Bayesian framework allows to incorporate multiple sources of information as a prior and any readjustment of the data/model hypotheses into the estimation process and produce a posterior zPDF.

In this work, we have demonstrated that by using a simple entry model and a few ML-algorithms that exploit descriptors of the redshift PDF, it is possible to capture an accurate description of the spectroscopic redshift reliability. This approach paves the way for fully automated processing pipelines of large spectroscopic samples as for next-generation large-scale galaxy surveys. We expect to further develop and test our method for the needs of the Euclid space mission when large simulations of realistic spectra become available. Advanced techniques in ML, such as neural networks and deep learning, will be explored to build a complex learning scheme.

Acknowledgements

We thank the referee for helpful comments and multiple suggestions that significantly improved this paper. This work is part of a thesis co-funded by the University Aix-Marseille and the Centre National d’Études Spatiales (CNES) in the context of the upcoming Euclid mission of the European Space Agency, as developed by the Euclid Consortium. The authorswould like to thank the members of the AMAZED team who provided helpful feedback and insightful discussions on this topic. This work benefitted from access to databases managed by the CESAM (Astrophysical Data Center of Marseille) in LAM (Laboratoire d’Astrophysique de Marseille). We would also like to express our thanks to all the contributors whose fruitful comments and revision of this manuscript helped us expand our understanding and improve the paper.

Appendix A Assigning probabilities

In the data, if the noise is assumed Gaussian, additive, and i.i.d., the data model for a single observation D_k is: $$\text{datum}{d}_k=\text{true}{x}_k+\text{noise}{n}_k;N_k~\mathcal{N}(0;{\sigma }_k).$$(A.1)

The variables d_k, n_k , and x_k are realizations of the random variables (r.v.) D_k, N_k and X_k .

By marginalizing over the r.v. X_k and N_k: $$p(D_k|z,M_t,I)={\displaystyle \int {\displaystyle \int p}}(D_k,N_k,X_k|z,M_t,I)\text{d}{X}_k\text{d}{N}_k.$$(A.2)

Assuming X_k ⊧N_k: $$\text{begin}align*p(D_k|z,M_t,I)=\text{ \&}{\displaystyle \int {\displaystyle \int p}}(X_k|z,M_t,I)\times p(N_k|z,M_t,I)\text{nn \&}\times p(D_k|N_k,X_k,z,M_t,I)\text{d}{X}_k\text{d}{N}_k.\text{nn end}align*$$(A.3)

From Eq. (A.1), the likelihood function is: $$p(D_k|N_k,X_k,z,M_t,I)=\delta (d_k-x_k-n_k)={\delta }_k.$$(A.4)

The Kronecker function δ_k implies n_k = d_k − x_k and allows to rewrite p(D_k|z, M_t, I) as: $$\begin{array}{lll}p(D_k|z,M_t,I)\hfill & =\hfill & {\displaystyle \int \text{d}}{x}_k{f}_X(x_k){\displaystyle \int \text{d}}{n}_k{f}_N(n_k){\delta }_k\hfill \\ \hfill & =\hfill & {\displaystyle \int f_X}(x_k)f_N(d_k-x_k)\text{d}{x}_k,\hfill \end{array}$$(A.5)

where f_X(x_k) and f_N (n_k) are the probability density functions of the random variables X_k and N_k.

If the true value X_k is considered as a deterministic variable: $$p(D_k|z,M_t,I)=f_N(d_k-x_k)=f_N(n_k)=p(N_k|z,M_t,I).$$(A.6)

Otherwise, the probabilistic model of x_k has to be integrated into the full expression of p(D_k|z, M_t, I).

The likelihood $\mathcal{L}(z,M_t)$ describes the probability of observing the full set of independent observations $D={\left\{D_k\right\}}_{k\in \Lambda }$ given a redshift z and a template model M_t and any additional information I.

Considering the aforementioned hypotheses on the data model, the likelihood is defined as following: $$\begin{array}{lll}\mathcal{L}(z,M_t)\hfill & =\hfill & p(D|z,M_t,I)=p(D_1,\dots D_n|z,M_t,I)\hfill \\ \hfill & =\hfill & p(N_1,\dots N_n|z,M_t,I)={\displaystyle \prod _{k\in \Lambda }p}(N_k|z,M_t,I)\hfill \\ \hfill & =\hfill & {\displaystyle \prod _{1\le i\le n}{(\sqrt{2\pi }{\sigma }_i)}^{-1}}\exp (-\frac{1}{2}{\chi }^2(z,t))\hfill \\ \hfill & \hfill & \text{vspace}-.5cm\hfill \\ {\chi }^2(z,t)\hfill & =\hfill & {\displaystyle \sum _{i=1}^n{\sigma }_i^{-2}}{[d_i-a_{\text{opt}}{t}_{i,z}]}^2,\hfill \end{array}$$(A.7)

where Λ is the wavelength range in use (with n datapoints), d_i and σ_i are the observed flux and noise spectra at pixel i, respectively, t_i,z is the redshifted template interpolated at pixel i, and a_opt is the optimal amplitude obtained from (weighted) Least-Square (LS) estimation.

We would like to point out that the estimation is in reality obtained from marginalizing over nuisance parameters θ, such as the amplitude A (r.v.) in the chi-square expression: $$\begin{array}{lll}p(z,M_t|D,I)\hfill & =\hfill & {\displaystyle \int p}(z,M_t,\theta |D,I)\text{d}\theta .\hfill \end{array}$$(A.8)

The joint-posterior PDF can be rewritten as: $$\text{begin}align*p(z,M_t|D,I)\text{nn \&}={\displaystyle \int \frac{p(\theta ,z,M_t|I)\times p(D|z,M_t,\theta ,I)}{p(D)}}\text{d}\theta \text{ notagnn \&}={\displaystyle \int \frac{p(z,M_t|I)\times p(\theta |z,M_t,I)\times p(D|z,M_t,\theta ,I)}{p(D)}}\text{d}\theta \text{ notagnn \&}=\frac{p(z,M_t|I)\times p(D|z,M_t,{\theta }_{opt},I)}{p(D)}\text{nnnnnotagnn \&}\times {\displaystyle \int p}(\theta |z,M_t,I)\frac{p(D|z,M_t,\theta ,I)}{p(D|z,M_t,{\theta }_{opt},I)}\text{d}\theta ,\text{nnend}align*$$(A.9)

where the highlighted integral in blue is usually approximated by a constant, and the computed likelihood in redshift estimation englobes the optimal estimation a_opt of the amplitude parameter in Eq. (A.7).

The amplitude a_opt is estimated at each trial (z, t): $$\text{begin}align*a_{opt}\text{ \&}={(t_z^{\top }w{t}_z)}^{-1}{t}_z^{\top }ws\text{ notagnn \&}=({\displaystyle \sum _{i=1}^n{s}_i}{t}_{i,z}{\sigma }_i^{-2})/({\displaystyle \sum _{i=1}^n{t}_{i,z}^2}{\sigma }_i^{-2}),\text{nn end}align*$$(A.10)

where $w=\text{diag}({\sigma }_1^{-2},\dots ,{\sigma }_n^{-2})$ is the weight matrix.

Appendix B ECOC for multi-class problems

The principle of ECOC (Error-Correcting-Output-Codes) is based on the binary reduction of the multi-class problem using a coding matrix $\mathcal{M}\in {\left\{-1;0;+1\right\}}^{K\times L}$ to design a codeword. $$\mathcal{M}=\text{nn begin}blockarraycccc\text{nn \&}{l}_1\text{ \& \&}{l}_L\text{nn begin}blockc(ccc)c_1\text{\&}{m}_{11}\text{\&}\cdots \text{\&}{m}_{1L}\text{nn \&}⋮\text{\&}\ddots \text{\&}⋮c_K\text{\&}{m}_{K1}\text{\&}\cdots \text{\&}{m}_{KL}\text{nn end}block\text{nn end}blockarray\text{nn vspace}-0.6cm,$$(B.1)

where:

• L: number of learners;

• K: number of distinct classes.

The codewords m_k = (m_k,1, …, m_k,L) translate the membership information for each class c_k given a binary scheme:

• m_kj = −1: c_k is the negative class for learner l_j,

• m_kj = 0: All observations associated with c_k are ignored by the learner l_j,

• m_kj = +1: c_k is the positive class for learner l_j.

Codewords are generated using existing coding strategies such as OVA (one-versus-all), OVO (one-versus-one) and dense random. Coding matrices for a example of a four-class problem are shown in Fig. B.1.

Fig. B.1 [Examples of ECOC coding design] . The black, white, and gray boxes refer respectively to mk,j = +1, –1 or 0.

Each learner l_j is associated with two superclasses, $\left\{S_{+};S_{-}\right\}$ referring to the positive and the negative classes, respectively, that are used to encode the response vector Y _train into a binary vector ${[Y_{\text{train}}]}_{\text{j}}$.

Trainingthe learner l_j with $\left\{X_{\text{train}};{[Y_{\text{train}}]}_{\text{j}}\right\}$ is performed with the usual classifiers such as SVM, MLP, and so on.

A class prediction for an unlabeled spectra x₀ in X_est is achieved in two steps:

• Step 1: each trained learner l_j provides a binary prediction: ${(y_0)}_j\in \left\{-1;+1\right\}$.

• Step 2: the bit vector y₀ for all learners is decoded into the initial K-class by minimizing a distance metric Δ as the Euclidean distance ${\Delta }_k={{\displaystyle \sum _{j=1}^L{(m_{kj}-{(y_0)}_j)}^2}}^{1/2}$ or a binary loss function ${\Delta }_k={\displaystyle \sum _{j=1}^L\left|m_{kj}\right|}g(m_{kj},s_j)$, where g is a binary loss function and s_j the scorefor learner j.

  The predicted class ${\widehat{c}}_k$ for x₀ is associated with the index k for which the vector Δ is minimal.

Appendix C Description of classification algorithms: SVM – Ensemble methods

Support-machine vectors

The SVM method classifies the data by finding the best hyperplane separating the datapoints of one class from those of another category.

Given a training set of M datapoints ${(x_i,y_i)}_{i\in 1\dots M}$, where x_i refers to the P-dimensional feature vector and y_i the associated label that indicates whether the datapoint belongs to the positive class (y_i = +1) or the negative class (y_i = −1), the objective of SVM is to separate the data into distinct classes using a separating rule in form of a parametrized function f(x ).

For linearly separable data, the equation of the hyperplane is: $$f(x)=f_{w,b}(x)=w.x+b=0,$$(C.1)

where the scalar product w.x is equivalentto w^⊤x.

An infinity of hyperplanes verify Eq. (C.1), but only one hyperplane maximizing the margins between the observationsand the hyperplane exists. This optimal hyperplane verify: $$\begin{array}{lll}\{\begin{array}{ll}(w.x+b)\ge +1\hfill & \text{if}{y}_i=+1\hfill \\ (w.x+b)\le -1\hfill & \text{if}{y}_i=-1\hfill \end{array}\hfill & \iff \hfill & y_i(w.x+b)\ge +1.\hfill \end{array}$$(C.2)

To find the“best” linear hyperplane minimizing the margins $2{(w.w^{\top })}^{-1/2}$, the SVM algorithm consists in solving a quadratic problem: $$\begin{array}{ll}\underset{w,b}{\text{minimize}}\hfill & \frac{1}{2}(w{w}^{\top })\hfill \\ \text{subjectto}:\hfill & y_i(w.x_i+b)\ge +1.\hfill \end{array}$$(C.3)

For non-linearly separable data, the use of a kernel φ trick enables to map the distribution of the datapoints x into a projected space where φ(x) can be linearly separable, and defines, in the same approach as in Eq. (C.2), a quadratic problem: $$\begin{array}{ll}\underset{W,B}{\text{minimize}}\hfill & \frac{1}{2}(W{W}^{\top })\hfill \\ \text{subjectto}:\hfill & y_i(W.\phi (x_i)+B)\ge +1,\hfill \end{array}$$(C.4)

where f(x) = f_W,B(x) = W.φ(x) + B = 0.

The selection of an adequate kernel K is determined by a list of criteria. By definition, a kernel must be symmetric, definite positive, square integrable and satisfy: $$\{\begin{array}{ll}\hfill & K_{i,j}=K(x_i,x_j)=\phi (x_i).\phi (x_j)\hfill \\ \hfill & \exists ({\lambda }_1\mathrm{...}{\lambda }_N)\in ℝ:{\displaystyle \sum _{i=0}^N{\displaystyle \sum _{j=0}^N{\lambda }_i{\lambda }_{j}K(x_i,x_j)\ge 0}}.\hfill \end{array}$$(C.5)

Among commonly used kernels:

• Linear: K(x_i, x_j) = x_i.x_j

• Power: $K(x_i,x_j)={(x_i.x_j)}^m$

• Gaussian (rbf): $K(x_i,x_j)=\exp (-\frac{1}{2}|x_i-x_j{|}^2/{\sigma }^2).$

Further details about the SVMs are available in Vapnik (2000) and Cristianini & Shawe-Taylor (2000).

C.2:

Ensemble classifiers

The principle of ensemble methodology is to combine a set of predictions from different learners in order to improve the accuracy of a single learner.

Among the ensemble methods, two distinct approaches are identified:

• 1.

  Averaging methods: Bagging/ Random forests...

• 2.

  Boosting methods: AdaBoost/GentleBoost/RSBoost/...

– AdaBoost

AdaBoost, known also as “Adaptive Boosting” refers to a specific algorithm of boosted classifier defined as the sum of individual predictions from T weak learners. The algorithm aims to minimize the (weighted) classification error at each iteration: $${\epsilon }_t={\displaystyle \sum _{i=1}^N{d}_i^{(t)}}\text{ mathbbm}1(y_i\ne h_t(x_i)),$$(C.6)

where:

• x_i is the feature vector of the ith observation.

• y_i is the true label of the ith observation.

• h_t is the prediction of learner t.

• 1 is the indicator function.

• $d_i^{(t)}$ is the weight of the ith observation at step t.

• t the iteration step from 1 to T.

At the first iteration, the weights $d_i^{(t)}$ are initialized (e.g., $d_i^{(t)}=1/N$) and the weak learner h_t is obtained by minimizing the error ε_t . For the next iteration, the weights of the learner (t + 1) are adjusted according to the performance of the previous one (t): whether increase $d_i^{(t+1)}$ for misclassified observationsby learner t, or reduce the weights otherwise. The learner h_t+1 is trained using the updated weights $d_i^{(t)}$ in the error ε_t+1.

After training, the prediction for a new data point, x, is obtained by combining the individual predictions of all weak learners: $$f(x)={\displaystyle \sum _{t=1}^{\top }{\alpha }_t}{h}_t(x);{\alpha }_t=\frac{1}{2}\log \left(\frac{1-{\epsilon }_t}{{\epsilon }_t}\right)\cdot$$(C.7)

The AdaBoost algorithm can also be viewed as a minimization of an exponential loss function: $${\displaystyle \sum _{i=1}^N{w}_t}\exp (-y_{i}f(x_i)),$$(C.8)

where w_t are normalized observational weights.

– LogitBoost

Following a similar approach to AdaBoost, the LogitBoost consists in training learners sequentially by minimizing an error function ε_t ; the only difference being the minimization of the error function with respect to a fitted regression model $\text{widetilde}y$ instead of y: $${\epsilon }_t={\displaystyle \sum _{i=1}^N{d}_i^{(t)}}{(\text{widetilde}{y}_i-h_t(x_i))}^2;\text{nn widetilde}{y}_i=\frac{y_i^{*}-p_t(x_i)}{p_t(x_i)(1-p_t(x_i))},$$(C.9)

where:

• $y_i^{*}$ are modified labels: $y_i^{*}=0$ if y_i = −1; and $y_i^{*}=1$ otherwise.

• p_t(x_i) is the predicted class probability for the ith observation to be in the positive class “+1” given by the learner t.

– GentleBoost

Also called Gentle AdaBoost, this algorithm combines the methodology of AdaBoost and LogitBoost. An exponential loss function is minimized with a different optimization strategy to AdaBoost. Further, similarly to LogiBoost, weak learners fit a regression model $\text{widetilde}y$ to the response variables y.

– Bagging

Bagging, referring to “bootstrap aggregation”, consists in generating m new training sets P_j , each of size N^′ , by uniformly sampling with replacement from the initial training set $I={(x_i,y_i)}_{i\in 1\dots N}$.

The m models are trained separately and the class prediction of an unlabeled data x is obtained by combining the individual predictions of the m models: “averaging” if regression, or “voting” if classification.

Further details on the ensemble algorithms can be found in Dietterich (2000).

Appendix D Measures for multi-class classification

For a binary classification, the confusion matrix represents the fraction of predicted labels versus the true classes. Four quantities are directly measured:

• TP: True Positives.

• TN: True Negatives.

• FP: False Positives.

• FN: False Negatives.

For multi-class classification, the approach consists in estimating these measures for each class. For example:

From the confusion matrix, the overall performances of the classification are quantified with the following measures:

Further details are provided in Fawcett (2006).

Appendix E Description of the FCM clustering algorithm

Similar to the k-means algorithm that aims to minimize the intraclass variance, the FCM (Fuzzy C-Means) algorithm exploits additional information about membership of the data to multiple clusters.

To partition a dataset $\overrightarrow{X}={(x_1\dots x_M)}^{\top }$ of P-dimensional vectors into K clusters, thealgorithm aims to solve a quadratic problem in order to determine the optimal solution (U, G), where G = (g₁…g_M) refers to the centroids of the final K clusters and $U={({\mu }_{ij})}_{\{\begin{array}{c}1\le i\le K\\ 1\le j\le M\end{array}}$ is a coefficient matrix of class memberships for each element. $$\begin{array}{lll}\underset{U,G}{\text{minimize}}\hfill & J(X;U,G)\hfill & \hfill \\ \text{subject to:}\hfill & {\displaystyle \sum _{i=1}^K{\mu }_{ij}}=1,\hfill & j=1,\dots ,M.\hfill \end{array}$$(E.1)

The algorithm proceeds iteratively and converges when the estimated coefficient matrix at the iteration t is not very different from its previous estimation: $$\text{begin{matrix}}\Vert U^{(t)}-U^{(t-1)}\Vert <{\epsilon }_0\text{ \&{},\text{where begininlinestripnssi438}\${\epsilon }_{\text{0}}$$(E.2)

Further, the elements ${(x_j)}_{j=1,\dots ,M}$ are said to belong to the class ${(c_i)}_{i=1,\dots ,K}$ for which the final coefficient μ_ij is maximal.

The matrix U can be further exploited to assess the membership level of the element x _j to its predicted class.

The cost function to minimize is: $$J(\overrightarrow{X};U,G)={\displaystyle \sum _{i=1}^K{\displaystyle \sum _{j=1}^M{\mu }_{ij}^m}}{D}_{ijA}^2,$$(E.3)

where the distance D_ijA, the coefficient μ_ij and the centroid g_i are defined as following: $$\begin{array}{lll}\hfill & \hfill & D_{ijA}^2=\Vert x_j-{\overrightarrow{g}}_i{\Vert }_A^2={(x_j-{\overrightarrow{g}}_i)}^{\top }A(x_j-{\overrightarrow{g}}_i),\hfill \\ \hfill & \hfill & {\mu }_{ij}={\left({\displaystyle \sum _{k=1}^K{D}_{ijA}^2}/D_{kjA}^2\right)}^{-2/(m-1)},\hfill \\ \hfill & \hfill & {\overrightarrow{g}}_i=\left({\displaystyle \sum _{j=1}^M{\mu }_{ij}^m}{x}_j\right)/\left({\displaystyle \sum _{j=1}^M{\mu }_{ij}^m}\right).\hfill \end{array}$$(E.4) (E.5) (E.6)

In the FCM,the fuzzifier parameter m ≥ 1 is used to determine the level of fuzziness: if m = 1, the coefficient matrix is binary, which is equivalent to a hard partitioning. Usually, in the absence of prior information about the datamodel, the value m = 2 is used.

For the norm $\Vert .{\Vert }_A^2$ in Eq. (E.4 ), a common choice for the matrix A is the identity matrix, but it can be designed to incorporate individual variances of the data as $A=\text{diag}({\sigma }_1^{-2}\dots {\sigma }_M^{-2})$ or the inverse of the covariance matrix.

Appendix F Additional tables

References

All Tables

All Figures

	Fig. 1 VVDS-Deep galaxy spectra with identifiable spectral features at known redshifts: emission/absorption line, D4000A break.
In the text

Fig. 2 Posterior redshift PDF of two VVDS-Deep spectra. Top: a unimodal zPDF characterizes a very reliable redshift measurement (a single peak at z=0.879). Bottom: a multimodal zPDF refers to multiple redshift solutions (multiple peaks) possibly with similar probabilities associated to a diminished confidence level of the MAP estimate at z = 1.159. The quantity ${\delta }_{{\Theta }_z}$ refers to thefixed step of the redshift grid used to compute the zPDFs.

In the text

	Fig. 3 Display of two zPDFs with multiples modes and a similar dispersion.
In the text

	Fig. 4 Display of two zPDFs with multiples modes and a different dispersion.
In the text

	Fig. 5 Display of two zPDFs with multiples modes and different characteristics of the two best z solutions.
In the text

	Fig. 6 Display of two zPDFs with a single mode and a different width of CR * .
In the text

Fig. 7 3D representation of the feature matrix X. Broadly, two categories are noticeable. The first group refers to the zPDFs with high dispersion, large zMAP peak and low probability P(zMAP|D, I) that can be assimilated to multimodal PDFs or platykurtic unimodal PDFs. The second group characterizes the zPDFs with medium-to-low dispersion, narrow zMAP peak and high P(zMAP|D, I) that depict strongly peaked unimodal PDFs.

In the text

	Fig. 8 Clustering the zPDFs features X in a dichotomized pattern. The clustering strategy exploits the classic FCM algorithm at each step to decompose the input data into two sub-classes using the entire set of descriptors. The final categories ${\left\{C_k\right\}}_{k\in \{1,2,3,4,5\}}$ are displayed in distinct colors.
In the text

Fig. 9 Representative zPDFs for each clusters. Display of representative zPDFs in each cluster obtained from clustering. The shift in the confidence level in the ${\left\{C_k\right\}}_{k\in \{1,2,3,4,5\}}$ clusters is apparent in the type of the zPDF: from multimodal zPDFs to unimodal zPDFs with narrower zMAP peaks, the confidence level ranges from “extremely unreliable redshift estimate” (C1) to “very certain redshift estimate” (C5).

In the text

	Fig. 10 Clusters repartition in a selected 3D space. The five zReliability clusters described in Sect. 4.3.1 are associated to different types of redshift PDFs, where the two extreme categories C1 and C5 , respectively,describe highly dispersed multimodal zPDFs and peaked unimodal zPDFs.
In the text