© ESO, 2016

1. Introduction

Red supergiants (RSGs) are evolved high-mass stars, characterised by very high luminosities log (L/L_⊙) ~ 4.5–5.8 (Humphreys & Davidson 1979) and late spectral types (K and M). They are the result of the evolution of moderately high-mass stars with masses from ~8 to ~40M_⊙, which represent the overwhelming majority of high-mass stars (Ekström et al. 2013). Since this phase is short (≲10% of their lifetime), evolutionary models for high-mass stars find a strict test-bed in the RSG phase.

The interest of RSGs goes beyond their role as evolutionary model constraints. Due to their high luminosity and low temperature, RSGs appear very bright in the infrared, and thus are easily observable at very large distances, even if they are affected by high extinction. Thanks to this, in the past few years, several massive and highly reddened clusters have been discovered in the inner Galaxy (for example Davies et al. 2007; Negueruela et al. 2012), in the region where the tip of the Galactic bar is believed to touch the base of the Scutum arm, revealing recent widespread massive star formation in this part of the Milky Way.

Stars massive enough to pass through the RSG phase are expected to end their lives as core-collapse supernovae (SNe). In fact, RSGs are the progenitors of type-IIP SNe (Groh et al. 2013; Smartt 2015), which is the most frequent SN type. Thus, the characterisation of RSGs (individually and as population) has an obvious interest for SN studies.

From a theoretical point of view, high- and intermediate-mass stars are easy to tell apart because of their very different evolutionary paths. High-mass stars are those with enough mass (≳8M_⊙) to end their lives as SNe after a few million years of life, while intermediate-mass stars will go through the asymptotic giant branch (AGB) phase, lose their envelopes, and finally become white dwarfs (except in a few cases, close to the limit with high-mass stars, where an electron-capture SN is possible).

Despite their different natures, RSGs are hard to distinguish from other late type stars, such as AGB or red giant branch (RGB) stars, by using only photometry. The intrinsic colours of all these stars are the same, as their temperatures are similar. Of course, the bolometric magnitude, M_bol, of RSGs is much higher than that of RGB and most AGB stars, but this is not really helpful when the distances and extinctions are unknown, and there are many less luminous but closer foreground stars, as is the case of the Galactic plane. To break this degeneracy, spectroscopic studies are necessary.

Classical spectral classification criteria were originally defined for the optical range (e.g. Turnshek et al. 1985; Keenan 1987, and references therein). However, as has been mentioned before, late stars are more easily accessible in the near infrared (NIR) than in the optical. The calcium triplet (CaT) spectral region, from ~8400Å to ~8900Å, has many advantages for a spectral study of RSGs. Firstly, this region is close to the emission peak of stars with temperatures typical of RSGs, and it is less affected by extinction than the optical range. Secondly, it is not affected by strong telluric absorption, as it is inside an atmospheric window. Thirdly, it is rich in spectral features that can be used for spectral classification, and many works have already studied them (Kirkpatrick et al. 1991; Ginestet et al. 1994; Carquillat et al. 1997; Munari & Tomasella 1999). In fact, the CaT itself is a well-known luminosity discriminator (e.g. Diaz et al. 1989).

There are still some unresolved issues related to the spectral type and luminosity classification of RSGs. The number of standard stars of the MK system classified as RSGs is very limited, and some of them are not very reliable standards because they present spectral variations of a few subtypes. Even more dramatic is the situation among the M-type RSGs: only a handful of them have been sufficiently well-characterised. Thus, the number of RSGs studied in works dealing with cool stars in general (e.g. Kirkpatrick et al. 1991; Ginestet et al. 1994; Carquillat et al. 1997) is really low, and they cover only the K sequence. In their spectral atlas of the CaT region at moderately-high resolution, Munari & Tomasella (1999) reach later spectral types, but only for dwarf and giant stars, never supergiants (SGs). In fact, the number of SGs considered by Munari & Tomasella is extremely low.

For M1 and later types, there are TiO molecular bands growing deeper in this wavelength range. Classical criteria define the M sequence by the presence and depth of the TiO bands in the optical spectral range. In the CaT range there are two main TiO bands (with bandheads at 8432 + 8442 + 8452Å and 8859Å), plus a weak VO band (at 8624Å), which are used for the same purpose (see Fig. 1). However, these bands represent a major complication for luminosity classification. At low and mid-resolutions these bands erode the continuum, weakening other spectral features, and even erasing them (Dorda et al. 2013). Therefore, the bands affect the line ratios and other measurements used as luminosity class (LC) criteria, rendering most of them useless except for the earliest M subtypes (those earlier than M3). Being so, extrapolation of the classification scheme from earlier types cannot be used. Even the CaT becomes unable to separate clearly LC I from LC II and LC III (Negueruela et al. 2011). However, the classification is still roughly possible if the spectral type (SpT) is known, as this will predict the TiO bandhead depths, and therefore warn about the erosion suffered by other spectral features (Negueruela et al. 2011, 2012).

Interest in the CaT range has grown in recent years because this is the spectral range that is being observed by the Gaia space telescope. Gaia uses its Radial-Velocity Spectrometer (or RVS) to observe the centre of the CaT range (from 8470Å to 8740Å) at medium resolution (R = λ/ Δλ = 11 500; Katz et al. 2004). Unfortunately, the spectral range observed does not cover any of the two main TiO bandheads in the region. Thus, for all M stars observed, all the spectral features in the RVS spectral range are affected by molecular band erosion, while the corresponding bandheads are not seen (and therefore their depth cannot be measured). In consequence, a strong degeneracy between SpT and luminosity appears for all bright late stars. Given the high extinction towards the inner Galaxy, many of the stars observed by the RVS in this area of the sky will be luminous cool stars. Because of their brightness in the CaT region, they will be excellent tracers of structure in these obscured regions. A good luminosity classification, however, will be necessary to make use of this information.

In this paper, we set out to derive spectral and luminosity classification criteria making use of the spectral features available in the spectra of cool luminous stars. These criteria will be useful for the analysis not only of Gaia spectra, but also of the products of forthcoming spectroscopic surveys, such as those that will be conducted with William Herschel telescope Enhanced Area Velocity Explorer (WEAVE).

2. Measurements

We have made use of a very large spectroscopic database, comprising stars from the Milky Way and both Magellanic Clouds (MCs). We show examples of the spectra used, indicating the features measured, in Figs. 1 and 2.

2.1. Data from the Magellanic Clouds

González-Fernández et al. (2015) have recently published one of the largest spectroscopic catalogues of cool supergiants (CSGs) to date, including samples from both MCs. From now on, we will refer to this work as GDN2015. The present work makes use of this catalogue to bring some light to the identification and classification of RSGs, because it is large enough to obtain statistically significant results about RSGs from both galaxies.

The sample from GDN2015 also has another critical feature for our work: all objects were simultaneously observed in both the optical and CaT spectral ranges. As the spectral and luminosity classifications were done using classical criteria in the optical range, the assigned SpT and LC are independent of the features in the CaT spectral range. A general description of the observation, reduction, and classification of the data from the MCs is presented in GDN2015. Here, we offer a brief summary of the data that have been used for the present work.

The data from GDN2015 were obtained with the fibre-fed dual-beam AAOmega spectrograph on the Anglo Australian Telescope. This instrument is capable of taking a simultaneous observation of two different spectral ranges. For the “blue” range, in the 2012 and 2013 campaigns, the 580V grating was used. For the “red” range the 1700D grating was used, providing a resolution R ~ 11 000 around the CaT. The 1700D grating has a 500Å wide spectral range centred on 8700Å, but the projection of the spectrum from each fibre on the CCD depends on its position on the plate, displacing the range limits up to ~20 Å. Therefore, it is not possible to define a precise common range.

GDN2015 observed the Small Magellanic Cloud (SMC) on three epochs (2010, 2011, and 2012) and the Large Magellanic Cloud (LMC) on two (2010 and 2012). However, here we are only using the data from SMC-2012 and LMC-2013. There are two reasons for this. Firstly, the spectral range observed in 2010 did not cover the TiO bandhead at 8859Å. Secondly, in 2010 and 2011 only previously known RSGs were observed. The surveys of 2012 and 2013, however, were exploratory, using a candidate selection plus a significant number of previously known RSGs – in fact, almost all the known RSGs observed in 2010 and 2011. Therefore mixing these epochs would lead to redundancy for the hundred previously known RSGs, while leaving out the 2012 or 2013 campaigns would exclude a large number of new CSGs.

The spectral and luminosity classifications were done using the optical spectra of the observed targets. There are a number of targets observed more than once on the same epoch, but all the spectra were classified independently without knowing their identities. GDN2015 used these redundant targets to estimate the internal coherence of the classification, finding that the mean differences between the spectral classifications of the redundant targets are about one subtype for the SpT and half a subclass for the LC. We use these values as uncertainties for the SpT and LC in this work. The results of this coherence test are detailed in Table 2 of GDN2015.

For this work, we have used all the stars from the 2012 and 2013 campaigns except the Carbon stars and those with SpTs earlier than G. We removed these stars because their NIR spectra are very different to our target late stars and thus they are easily identifiable. Nevertheless, we kept late-type, non-supergiant objects, because these stars passed the cut of the photometric criteria, and therefore represent the kind of interlopers to handle in a survey looking for RSGs. Therefore any useful spectral criteria should be able to separate these stars from the CSGs. These interlopers are mainly foreground stars (G, K and early-M dwarfs and giants), with a smaller number of AGB stars (high-luminosity and late-SpT giants) from the MCs themselves (but mainly from the LMC). Table 1 contains a summary of the data used for the present work.

2.2. Data from the Galaxy

We decided to complement the GDN2015 data with stars from the Milky Way, for two main reasons. Firstly, although GDN2015 provides a statistically significant number of CSGs, the number of mid- and late-M stars in this sample was small. These subtypes are not as frequent in the MCs as they are in the Milky Way, because CSGs tend to have earlier SpTs at lower metallicities (Elias et al. 1985). Secondly, we wanted to include objects covering a broad range of metallicities, so that we could understand the effect of chemical composition on our criteria. In addition, a sample at approximately solar metallicity is needed to check the validity of our criteria for the range of metallicities that we may expect to find among the stars that Gaia will observe towards the Inner Galaxy. Thus, we have observed a significant number of MK standards (or at least stars with spectral classification determined by Keenan & McNeil 1989, presenting LCs from I to III), and other well known CSGs in the Perseus arm. All these objects have spectral and luminosity classifications that have been repeatedly confirmed in the literature. We therefore considered them well characterised and did not perform our own optical classification. Table 1 has a summary of the data used for the present work, but the details about the observed Galactic stars are shown in Table B.1.

The stars from the Milky Way were observed along four different campaigns, one in 2011, two in 2012, and one in 2015, using the Intermediate Dispersion Spectrograph (IDS) attached to the 2.5 m Isaac Newton Telescope (INT) in La Palma (Spain). We used grating R1200R with the RED+2 CCD mounted with its 4096 pixel axis along the wavelength direction. This configuration covers a 572Å wide spectral range centred at 8500Å (i.e. the spectral range around the CaT), with a resolution R ~ 10 500, very similar to the resolution of the data from GDN2015 (R ~ 11 000).

IDS is a classical long-slit spectrograph, and so reduction of these observations was carried out in the standard manner, using the iraf facility¹. As a last step, we obtained velocities along the line of sight for these objects. This calculation follows that outlined in Negueruela et al. (2012), with only one particularity: to correct the wavelength calibration from instrumental flexures, we use the subtracted sky spectrum to cross-correlate the sky emission lines between objects. By doing this, we guarantee that all our measurements will have the same instrumental signature that can then be corrected by comparing with velocity standards.

2.3. Spectral features measured

To characterise the CaT region we selected the main atomic and molecular features present along the common spectral range that we have for all stars (from 8450Å to 8870Å), including known SpT and LC indicators, but also those not used previously. In total, we measured 47 atomic features and four molecular bandheads. See Tables C.1 and C.2 for the complete and detailed list.

The first step to measure these spectral features was to re-sample the spectra from each instrument to the same uniform resolution: R ~ 10 000, by convolving the spectra with a Gaussian kernel. Later, we compared the measurements done with and without the convolution, and we found the mean difference for each measurement to be smaller than the corresponding typical uncertainty. We conclude that this convolution from R ~ 11 000 and R ~ 10 500 to R ~ 10 000 does not affect significantly our results and, in consequence, future works using a similar resolution (as is the case of the Gaia RVS) may utilise our results without the need to re-sample their spectra to R ~ 10 000.

The second step was to shift each spectrum to their rest wavelength, using the previously calculated radial velocities. The third step was to measure the spectral features. This was done differently for the two types of features present in our spectra, atomic lines and molecular bandheads.

2.3.1. Atomic lines

For the atomic spectral lines, we decided to measure their equivalent widths (EWs). Previous works (e.g. Kirkpatrick et al. 1991; Ginestet et al. 1994; Cenarro et al. 2001, and references therein) proposed different methods for this calculation, through an estimation of the true continuum. However, because of the effect of the TiO bands over the continuum, these methods become useless for mid- and late-M stars. As we want uniform EW measurements for the whole SpT range, from early-G to late-M, we use local pseudo-continua to calculate the EWs. Table C.1 shows the ranges used for each line.

To define the local pseudocontinuum for each line, we studied the behaviour with SpT of its neighbouring spectral region. For each line we selected two spectral ranges, one to the red and one to the blue of the line, not affected by atomic lines at any of the SpT in our range (from G to M). Then for each line we calculated the linear regression with wavelength of its two pseudo-continuum ranges, and from it we obtained its EW. This method using local pseudo-continua does not measure a true EW, because we are not necessarily using the real continuum. With it, we only measure the apparent strength, at this resolution, of the line at a given SpT. Uncertainties on the EWs were calculated using the method proposed by Vollmann & Eversberg (2006).

The atomic feature centred at 8468Å (which spans from ~8462Å to ~8474Å) is a blend of many lines (mainly Ti i and Fe i) and multiple molecular bandheads of CN (Ginestet et al. 1994; Carquillat et al. 1997). It is considered a powerful luminosity indicator for stars with SpT earlier than M3 (Negueruela et al. 2011), but it is also sensitive to chemical anomalies (Carquillat et al. 1997). The measurement of this feature was not easy, because the continuum on its blue side changes strongly with SpT due to the presence of molecular bandheads and atomic lines at different SpTs. This feature, however, is too important to be discarded. Thus, we selected two continuum ranges on its blue side, one useful for early subtypes and the other for the M subtypes. We calculated two EWs for this feature in each case but then we kept only that with the slope of its regression closest to 0, as this is the measurement less affected by changes in the continuum.

2.3.2. Molecular bandheads

The rise of a molecular band changes drastically the shape of the nearby spectral range, creating a shoulder at the wavelength of the bandhead. For wavelengths redder than this, the apparent continuum is not flat, but has a positive slope, tending to the original continuum level, giving the whole molecular feature the shape of a saw tooth.

We defined two criteria to detect the presence of a bandhead. The first criterion is to test the shape of the bandhead. For this, we take a small spectral range of a certain width, centred on the bandhead wavelength. To accept the presence of a band, the maximum intensity in this range has to be to the blue of the minimum intensity. The second criterion requires that the difference between the maximum and minimum intensities within the range defined for the previous step is larger than the standard deviation of the intensity in the continuum. With this criterion we make sure that the bandhead has a significant depth.

If a given bandhead passes both criteria, we measure its depth, if not, we assign a value of 0 to it. For this step, we defined for each molecular band a bandhead central wavelength at the point where the bandhead starts (i.e. the point where the flux suddenly decreases), and a pseudo-continuum range bluewards of it. We also defined a small range to the red of the bandhead centre, where the minimum flux of the bandhead should be. The values defined for each bandhead are given in Table C.2. For the measurement, we first find the point where the flux is the lowest in the range defined to the red of the centre of the bandhead. This is the bandhead bottom (BB). After this, we perform a linear regression fit to the pseudo-continuum range, because we want to take into account the slope of the pseudocontinuum, and we extrapolate the resultant fit line over the defined centre of the bandhead. This value is the bandhead top (BT). From these values, we obtain the bandhead depth (BD): $\begin{array}{ccc}\mathit{BD}\mathrm{=}\frac{\mathit{BT}\mathrm{-}\mathit{BB}}{\mathit{BT}}\mathrm{·}& & \end{array}$To measure the uncertainty in a band’s depth, we use a method analogous to that of Vollmann & Eversberg (2006), but with two differences. We start from the expression for our BD, and we measure the signal-to-noise (S/N) of the pseudo-continuum using the residua from the fit to the pseudo-continuum instead of using the differences to the mean flux value of the pseudo-continuum. We do this because the pseudo-continuum is slopped by the effect of other molecular bands to its blue side, making the difference to mean flux meaningless. The resultant expression is: $\begin{array}{ccc}\mathit{\sigma }\mathrm{\left(}\mathit{BD}\mathrm{\right)}\mathrm{=}\frac{\mathrm{1}}{\mathit{S}\mathit{/}\mathit{N}}\sqrt{\frac{\mathit{BB}}{\mathit{BT}}\mathrm{+}{\left(\frac{\mathit{BB}}{\mathit{BT}}\right)}^{\mathrm{2}}}\mathrm{·}& & \end{array}$

3. Analysis

3.1. Principal component analysis: calculation and applications

Principal Component Analysis (PCA) is a statistical method that finds the directions of maximum variation in the multidimensional space of the input data. These directions are the Principal Components (PCs), determined by a list of coefficients, that is, the coordinates of these directions in the input data space.

The vast majority of our stars have no emission lines. In the few RSGs and AGBs which present this effect, they are weak, with the line profiles dominated by the absorption components. Therefore, there are no stars in our sample with negative EW values because of emission. Such values come exclusively from lines that disappear almost completely because of the presence of molecular bands, but also when the atomic line is too weak to be measured (in most cases because the star has a spectral type too early to display it). So, we have assigned a value of 0 to all the negative EWs for the PCA calculation.

We performed the PCA through a bootstrapping process, as a way to insure that the results obtained would not be driven by the stochasticity of a particular sample. For this, we first took a random subsample of 500 stars from our whole sample of almost 900 stars. Then we performed the PCA for this subsample. This process was repeated 10 000 times. In this way, for each PC coefficient we obtained a distribution of values, taking its median as the final value, and having its standard deviation as a measure of its reliability.

In a first study we used all the EWs and bandheads detailed in Tables C.1 and C.2. We tried then a shortened line list, including only those lines whose identification is certain, finding that it leads to less noisy results. This reduced list contains all the lines of Ca ii, Fe i, Ti i, Mn i, Si i and Mg i, except those marked with “?” because we are not sure about their chemical species. We also removed Fe i 8621.5Å and Ti i 8623Å and the VO bandhead at 8624.5Å, because they may be affected by the Diffuse Interstellar Band (DIB) at ~8621Å, in an unpredictable way. The exact position of the DIB in the spectra depends on the relative radial velocities between the interstellar medium that generates the DIB and the observed target, and thus it may appear at different positions around ~8621Å. The depth of this DIB depends on the amount of extinction to the target, which is not a problem for our sample, but will be for other samples affected by moderate or high extinction. In total, our shortened list contains 29 atomic lines and three bandheads.

Because of the nature of the PCA, the first few PCs contain most of the variance, while later PCs contain progressively less, as shown in Fig. 3. That is why the standard deviation values for the coefficients of PC6 and later are significantly higher. In our PCA, the principal components PC1, PC2, and PC3 together contain more than 80% of the accumulated variance. To reach 98%, the first 15 PCs are necessary. Therefore we only show here the first three PCs, which display the clearest correlations with SpT and LC (see Figs. 4 and 5). Tables D.1–D.3 show the linear combinations through which PCs were calculated.

PC1 has a clear relation with both LC and SpT (see Fig. 4). For early SpTs, the SpT distribution of the SGs has a negative correlation with PC1 down to M2, where the minimum value of PC1 is reached. Non-SGs have a similar behaviour, forming a strip parallel to the SGs. From M2 toward later subtypes, both SGs and non-SGs are mixed in the same strip, which has a positive correlation between PC1 and SpT. The change in the behaviour of PC1 around M2 may be caused by the appearance of molecular bands from approximately M2 onward (see Sect. 4.3.4). As these bands grow in depth, the atomic lines become progressively weaker due to the erasure of the continuum.

PC2 has an easier interpretation (see Fig. 5). It shows a very clear correlation with SpT for all SGs, and it seems not to be affected by metallicity, as the stars from different galaxies present the same behaviour. The separation between early SGs and non-SGs is less clear than in PC1, but still significant for SpT earlier than late K subtypes.

We can divide PC2 into three regions regarding its relation with SpT. The first is for subtypes earlier than approximately M2, with a negative slope. The second range goes from M2 to M7, with a more markedly negative slope. Finally, for stars M7 or later, which are poorly sampled, the slope seems to become positive. The two changes are caused by the behaviour of TiO bands. As with PC1, the change at M2 is because this is when the TiO bands first appear. The break at M7 is because the TiO band at 8432 Å has become so strong that it is affecting the depth of the other bands, which probably are saturated: instead of growing, the bands become weaker for later SpTs (see Sect. 4.3.4), while atomic lines have almost disappeared.

3.2. Identifying supergiants

3.2.1. Principal components

In order to generate diagnostics, we have combined the three first PCs in diagrams, because each one contains significant information about the LC and SpT.

In Fig. 6 we show the (PC1, PC2) diagram. The data follow a clear sequence that depends mainly on SpT, because of the behaviours described in Sect. 3.1. This diagram is very informative and it can give a rough estimate of the SpT and LC of a star, by simply looking at at the star’s position on the diagram. In addition, we have to note that all the stars follow the same trends regardless of their galactic origin (i.e. their metallicity).

The (PC1, PC3) diagrams (Fig. 7) provide a very good way to separate the SGs from the non-SGs (see Fig. 7), by simply tracing a line between both groups. If we apply this criterion, we find that almost all the stars that are not classified correctly have SpTs later than approximately M3 (see Fig. 7, panel b). This is because the mid and late M type SGs tend to be in the same areas of the diagram as the non-SGs. Therefore, the number of wrongly classified stars can be reduced by first using the (PC1, PC2) diagram to flag out these late stars.

In conclusion, these diagrams provide a quick and easy way to obtain a preliminary SpT and LC classification, especially for stars earlier than M3, but they do not represent a very accurate method.

3.2.2. Support vector machine classification

To calculate the optimal boundary between different groups of stars in the 15-dimensional space of the PCs, we used the Support Vector Machines (SVM) method, through the python package sklearn (Pedregosa et al. 2011). This method requires a sample of stars labelled as belonging to each different group, to train the SVM. As we decided to use a linear version of the SVM, its output result is a vector of coefficients, which in linear combination with the 15 PCs produces a hyper-plane (in 15 dimensions) as a boundary between the two different groups labelled. We repeated the SVM through a bootstrapping process 10 000 times, each time giving as input a randomly selected sample with a size N depending (about 50% of the size) on the corresponding input sample.

As the mid and late M type stars have a different behaviour from the rest of the sample, they are the main source of SG misidentification. In order to minimise these errors, we split our sample through the SVM process into two sets, according to whether the stars are earlier or later than a given subtype (K5, M0, M1, M2, and M3), and then, separate the SGs from the non-SGs within each of these sets, using again the SVM method. This system requires us to determine which is the optimal M subtype boundary that minimises the wrong identifications. We could not use subtypes later than M3 as the boundary because the late subsample then becomes too small to perform any meaningful analysis. The calculated coefficients that separate the indicated groups are given in Tables E.1–E.3.

3.3. Fitting spectral types

The spectral differences between adjacent subtypes are too subtle to allow separation through the SVM method, because the dispersion in values of most spectral features within a given subtype is significantly high (see figures in Sect. 4.3). In view of this, we decided to complement the PC analysis with studies of the behaviour of the individual spectral features that show the stronger trends with SpT, and performed linear fits to their variation with SpT. We also performed linear fits to the behaviour of the PC2 with SpT. The linear regressions were performed using the Theil-Sen estimator (which is based on a median of the slope distribution obtained through random pairings of the sample) to avoid the effect of the outliers.

As with the PCs, most spectral features change their behaviour as we move to later SpTs, because of the rise and saturation of the TiO bands. Therefore, we divided the sample into two different subsamples, one composed of those stars considered early, and the other by those considered late, because we expected different behaviours for each one of them. The subsamples were obtained by using the hyper-plane calculated with the SVM, taking as boundary the M1 subtype (because the main TiO bands in the CaT spectral range become measurable at this subtype). Afterwards, we followed a different procedure for each group.

3.3.1. Early subsample

For the early subsample, we identified two indicators that correlate well with SpT. Firstly, the strength of Ti i lines (at 8518.1, 8683.0, 8692.0, 8730.5, and 8734.5Å), as seen in Fig. 8), most likely because they are very sensitive to temperature (Dorda et al. 2016). Thus, we have added together their EWs in a single value that we name “Ti index”. Secondly, PC2 shows a clear trend with SpT for early-type SGs. In both cases we have considered only the SGs as selected by the PCA-SVM method.

As the Ti index depends on the metallicity of the host galaxy, we have calculated two different fits, one for SGs from the SMC and the other for those of the LMC and the Galaxy. We have considered these together because the number of SGs earlier than M1 in these galaxies is low, and the effects on the observed spectra of the difference in metallicity between these two galaxies are small (see Fig. 2). The coefficients of the lines calculated are given in Table 4.

3.3.2. Late subsample

The late subsample was split again to remove the latest M stars, because their behaviour is different due to the saturation of the TiO bandheads. As their number is low, we were not able to use the SVM method. In its place, we separated them by simple cuts in the (PC1, PC2) diagram (see Fig. 6a): PC1 > 8 and PC2 <−3. As they are very few (only 16, and all non-SGs), no reliable analysis was possible for these very late-M stars. This left us with a sample of stars with spectral types between M1 and approximately M7.

Again, we found two indicators that can be used for the linear regression: PC2 (see Fig. 5), and the TiO bandhead at 8859Å (see Fig. 13). Both SGs and non-SGs present the same behaviour, and we used both SGs and non-SGs for these calculations.

In both cases (PC2 and TiO bandhead), the data distribution suggests that the regression should have a higher order than a simple linear fit. Therefore, we tested polynomial fits with order two and three for both variables in addition to lines calculated through Theil-Sen. These fits naturally reproduce the data better, yet the method to avoid overfitting used by Asensio Ramos (2006) shows that, given the uncertainties, these higher-order polynomials are not better fits than a first order polynomial. Therefore, in Table 4 we show the coefficients for the linear fits only.

4. Results

In Sect. 3, we developed a method to separate SGs from non-SGs using the PCA and SVM algorithms, and calculated linear fits that can automatically indicate the SpT of cool supergiants. In this section, we discuss the efficiency of the methods proposed and compare them with traditional spectroscopic criteria.

4.1. Efficiency in finding supergiants

We have developed a method to separate SGs from non-SGs using PCA and SVM algorithms, and calculated five sets of results. Each of these sets corresponds to the five different putative boundaries (at K5, M0, M1, M2, and M3; see Sect. 3.2.2) selected to separate the stars into the early and late subsamples. The results of this process are shown in Table 2. We define the efficiency as the fraction of known SGs in the sample or subsample that are tagged as SGs by a given criterion. The contamination is the fraction of stars identified as SGs that in fact are not. As efficiency and contamination are fractions, their two-sigma uncertainties are equal to $\mathrm{1}\mathit{/}{}^{}\sqrt{\mathit{n}}$, where n is the total number of SGs in the correspondingly sample.

The efficiency for the early subsample is extremely high: 0.99 for all the five boundaries, with uncertainties between ± 0.04 and ± 0.06 depending on the boundary. The efficiency is also high for the late subsample, with values between 0.89 and 0.97. The uncertainties of these fractions depend (inversely) only on the number of objects forming the group considered. Because the number of objects with late subtypes is lower, the uncertainties for the late subgroup are higher. As a result, the difference between the efficiencies for the early and the late subsamples are not statistically significant. In fact, the efficiency for the whole sample is always ~0.98 ± 0.04 for any of the five putative boundaries studied.

We have also evaluated the contamination in the five sets. For the whole sample the contamination is very low (0.02 ± 0.04). As in the case of the efficiency, the differences between the contaminations for both subsamples with all the boundaries used are not statistically significant.

These results show that the methods proposed are very effective, especially if we consider that they were obtained for a sample with a wide range of SpTs coming from three different galaxies. For completeness, we also checked if there are significant variations in the contamination and the efficiency for a mono-galactic sample. As we are planning to apply this method to a large Galactic sample, we measured the efficiency and contamination of our method using a sample composed only of stars from the Milky Way (i.e. our MK standards and stars from the Perseus arm). The results are shown in Table 3. Unfortunately, as this sample is smaller than the full sample, the uncertainties of the fractions are higher than those obtained for that sample. Thus, even though we find lower values for the efficiency (0.90 ± 0.13) in this case than for the full sample, the difference cannot be considered significant. The contamination obtained (0.03 ± 0.13) is also compatible within the uncertainties to that obtained for the full sample.

4.2. Spectral subtype estimation

For each of the regressions discussed in Sect. 3.3, we have calculated the distribution of the errors (i.e. the difference between the real value of the SpT and the value predicted by the regression). As the data have some outliers, the standard deviation was done through the median absolute deviation, a robust estimator. The results for each regression are shown in Table 4.

We divided the sample into two subsamples, early and late, before carrying out the fits. These divisions were made by applying the SVM coefficients obtained in the previous section, and present a small degree of contamination. Therefore, we show each linear regression covering all the subtypes in the subsample used, even if they are beyond the limit of the split. When this method is applied to a problem sample, the same may happen: a few stars will be in the wrong subsample, and the behaviour of such stars affects the linear regressions calculated. Thus, despite the “split”, each regression may give the classification of some points above or below the limit of the split.

In addition, there are a few very luminous stars that are outliers in the PC2 diagrams. The linear regression gives these stars SpTs earlier than G0, while they are really G or K hypergiants (or very luminous supergiants). We recommend the use of the Ti index instead of the PC2 for them. To identify these stars, the CaT can be used because these extremely high-luminosity stars have very high values of the CaT (see Sect. 4.3.1).

The SpTs of the late subsample are strongly correlated with both variables considered, the classical indicator (the strength of the TiO bandhead at 8859Å) and the PC2. The results prove that, for a quick spectral identification, PC2 is as good as the TiO bandhead. The standard deviations of these regressions are 0.7 subtypes, and so this classification is only slightly worse than the classical classification process by visual inspection of the whole spectrum (0.5 subtypes). The Ti index cannot be used effectively with the late subsample, because at the typical SpTs the Ti lines are heavily affected by molecular bands.

For the early sample, the fit of the SpT to the PC2 has a standard deviation of 1.7 subtypes, significantly larger than for the late subsample. The Ti index provides similar results, but it is affected by metallicity, and thus the calculated fits can only be used for populations with similar metallicities to the SMC or the Galaxy/LMC. Thus we obtained two different linear regressions, one for stars from the SMC and another one for those from the LMC. The results are better than for the PC2 linear regression, but the standard deviation is still significantly higher than for the late subsample. Thus, these regressions are merely a quick estimator. A detailed classification of the stars from the early subsample can only be performed using the classical criteria, as they are not affected by the appearance of TiO bands.

4.3. Classical criteria revisited

The two main features used to identify SGs are the blend of Ti i, Fe i, and CN molecular bands at 8468 Å, and the infared CaT (formed by the lines at 8498, 8542, and 8662Å). Ginestet et al. (1994) show the typical EW values they obtained for these features, but their SG sample is small and only with SpTs earlier than K3. Negueruela et al. (2011) also show both values for a small number of standards with SpTs from K0 to M5. However, the number of SGs in their sample is limited. Therefore we have decided to take advantage of our very large sample, and we show here the behaviour of both spectral features, in Figs. 9 and 11.

We also study one of the most useful ratios in this region, Fe i8514Å to Ti i8518Å, which was proposed as a luminosity indicator by Keenan (1945), and the behaviour of the TiO bandhead at 8859Å because this is the main TiO band in the region and has been widely used to obtain the SpT in the M sequence (e.g. Ramsey 1981; Negueruela et al. 2011).

4.3.1. Calcium triplet

The CaT depends strongly on luminosity (e.g. Diaz et al. 1989, and references therein), and it has been used extensively for luminosity classification (e.g. Ginestet et al. 1994; Carquillat et al. 1997; Negueruela et al. 2011). However, our values are systematically lower than those of these researchers’ (up to ~2–3Å with respect to those obtained by Ginestet et al. 1994). This is due to two reasons. Firstly, the continuum is determined differently. The method used by Ginestet et al. (1994) results in values of EW higher than in any other work. Secondly, for the spectral range that we have in common with Ginestet et al. (1994), almost all our stars come from the MCs, while all their objects come from the Milky Way. Because of the lower metallicity in the Clouds, our values are systematically lower.

The EW of the CaT depends also on SpT, as was shown by Ginestet et al. (1994). We find the same behaviour here (see Fig. 9), with the EW of the CaT growing from early-G subtypes, until it stops increasing around spectral type M0 due to line saturation at low temperatures. After this, it starts to decrease with SpT, due to the appearance of TiO bands, which quickly affect the continuum, erasing the wings of the CaT. In consequence, it becomes useless as a luminosity criterion for subtypes later than approximately M3.

In some works (Ginestet et al. 1994; Carquillat et al. 1997), the sum of the EWs of the three Ca ii lines were used, while in others (Diaz et al. 1989; Negueruela et al. 2011) only the two strongest lines (Ca ii 8542Å and Ca ii 8662Å) were considered. In both cases, in order to used the CaT as an LC discriminator, metallicity and the SpT range must be carefully considered, as the CaT changes noticeably with these two variables. We note that metallicity not only affects the strength of these lines, but also determines the typical SpT range of cool SGs. If we investigate the EW of the CaT for stars presenting a wide range of metallicities and SpTs, the separation between SGs and non-SGs seems to be around ~9Å (Fig. 9), with all SGs except some of those later than M3 having CaT with values higher than this. With this boundary, the efficiency is 0.95 ± 0.04, and the contamination is 0.07 ± 0.04 (there are some bright and/or high-metallicity giants with CaT values as high as 11Å). While efficiency is statistically similar to that obtained with the PCA method (see Sect. 4.1), the contamination is slightly, but still significantly, higher. However, the simplicity of this criterion makes it a good tool for a quick and preliminary classification.

Negueruela et al. (2011) proposed a boundary value of EW = 9Å for the sum of the two strongest lines of the CaT for stars from our Galaxy (at the typical metallicities and SpTs). We compare this boundary with our sample in Fig. 10. This criterion keeps the contamination at similar value as the PCA/SVM method (0.02 ± 0.05), but with a much worse efficiency, only 0.79 ± 0.04.

Finally, we checked those stars whose CaT values are much higher than typical for their SpTs. These extreme values are not caused by the measurement method, but are genuine physical features. Most of the stars with very high values are classified as Ia, but there is also one star (RW Cep) that is a known hypergiant (0–Ia). Thus, their extraordinarily high CaT values are likely due to their extreme luminosities alone. Moreover, most of these stars have G or early-K subtypes, significantly earlier than those usual for their galaxy. Therefore, this group seems to be composed of yellow hypergiants, whose properties are expected to be different from those of the typical RSGs. We will explore the behaviour of these stars in future works.

4.3.2. Blend at 8468 Å

The spectral feature at 8468 Å is a blend of many lines (mainly Ti i and Fe i) and multiple molecular bandheads of CN (Ginestet et al. 1994; Carquillat et al. 1997). It has been used as a luminosity indicator by Ginestet et al. (1994) and Negueruela et al. (2011), but with very limited samples.

In Ginestet et al. (1994) the blend shows a strong dependence with SpT from G0 to early-K, both for giant and SG stars, that becomes less pronounced for K stars. The blend in Negueruela et al. (2011) shows the same trend for K subtypes, and then becomes flat for early-M subtypes, surely because of the effect of TiO bands. The behaviour we show here (Fig. 11) is consistent with these two studies. However, our values are systematically lower than those from Ginestet et al. (1994), as was the case for the CaT, and for the same reasons.

In Negueruela et al. (2011), the value of EW = 1.2Å was used as a boundary between giants and SGs for Milky Way stars. As we show, this value seems appropriate for subtypes between mid-K and early-M only. However, this criterion should be carefully reconsidered for stars from low-metallicity galaxies, because their atomic features are weaker and also because the SG-typical SpTs are earlier.

Using the EW value as discriminator has a global efficiency of 0.46 ± 0.04 and a contamination of 0.13 ± 0.06. If we only consider those stars that are K4 or later, the contamination does not change (0.13 ± 0.06), but the efficiency becomes higher (0.76 ± 0.06). However, these values are still clearly worse than utilising the CaT or our PCA method (see Sects. 4.1 and 4.3.1).

4.3.3. Fe i 8514 Å and Ti i 8518Å

The ratio between the Fe i8514Å and Ti i8518Å lines was proposed as a criterion for LC classification by Keenan (1945), and it has been used with good results by Negueruela et al. (2012) for K and early-M stars. However, it has not been properly characterised with a statistically significant sample. Threshold values of the ratio have not been given before, either.

Figure 12 shows the relation between both lines, and how their values separate very effectively different LCs, though with a strong dependence on SpT. For early G types these lines are not useful because they are too weak, and for later stars these lines appear on the red side of a TiO bandhead (at 8513Å), and become useless as luminosity indicators around M2. Because of these reasons, we measure negative values for the EWs of these lines in some stars beyond these limits.

We have used the SVM method explained above (with 10 000 random subsamples of 500 stars each) to calculate the optimised line (and its uncertainties) that separates the SGs from the non-SGs based solely on these two lines. The coefficients obtained for this line (EW(8514.1) = m· EW(8518.1) + n) are m = 0.37 ± 0.05 and n = 0.388 ± 0.011. The efficiency of this boundary is 0.93 ± 0.04 and the contamination is 0.07 ± 0.04: both slightly, but significantly, worse than those obtained through the PCA method (see Sect. 4.1). However, the use of this line ratio has a disadvantage that we need to keep in mind: it depends only on two lines, neither especially strong. So, if either of these two is slightly anomalous, it will surely result in a wrong classification. The PCA method is more robust, as it depends on a large number of spectral features.

4.3.4. TiO bandhead at 8859Å

The spectral sequence for K and M subtypes in SGs is defined by the appearance and depth of TiO bandheads in the optical range (see GDN2015 and references therein), which are correlated with TiO bands in the CaT spectral range. The triple TiO bandhead at 8432 + 8442 + 8452Å together with the TiO bandhead at 8859Å are the strongest bandheads near the CaT, and hence the main SpT markers for the M sequence in this spectral region.

In Fig. 13, we can see that stars earlier than K5 present values close to zero. The TiO band first appears at approximately M1, and its strength increases with SpT until approximately M7, where it reaches a maximum. For later types, the band is saturated and the apparent depth of the bandhead starts to decrease, as the continuum is eroded by the TiO bands at shorter wavelengths. This behaviour does not seem to depend on LC, at least not between I and III, even though the number of confirmed RSGs with SpTs later than M6 is very small.

Although 8869Å TiO band provides information about the SpT of the stars, it cannot be used alone to assign a SpT. Late-M types may be confused with mid-M stars if the only SpT criterion is the strength of the TiO band because of the saturation of the band. Even more importantly, such stars are scarce in our sample, but they are quite common in the Galactic Plane (where many AGB stars have spectral types in the M7–M10 range). In addition, all the G and K stars could be confused with the early-M types, especially for noisy spectra.

These problems may be solved easily by attending to the atomic lines. For example, stars later than M8 hardly have any atomic line in the whole CaT spectral region, and they present a strong VO 8624Å bandhead. However, these criteria must be considered carefully, case by case, because the strength of the lines depends on their luminosity, and the VO bandhead may be contaminated by the nearby DIB at ~8621Å. In consequence, for a quick and reliable analysis of a large number of stars we suggest using the PCA detailed in Sect. 3.3.

4.4. Detecting extreme red supergiants affected by veiling

The veiling effect was first reported by Merrill (1940) in late stars, but Humphreys & Ney (1974) studied it carefully in SGs. The cause of the veiling is still unclear, but it has been suggested that it might be caused by free-bound emission (Humphreys & Ney 1974) or by scattering through an expanding circumstellar dust shell (Massey et al. 2009). The veiling weakens all absorption atomic lines significantly, but Humphreys & Ney (1974) showed that this weakening is wavelength-dependent for RSGs, with the maximum weakening around the CaT region. The EW of the CaT lines decreases below the typical EW values of red giants, reaching values similar to those of dwarf stars. However, the strength of the TiO bands does not seem to be affected.

Among SGs, it has only been reported for some extreme RSGs (ERSGs). These stars, of which only a handful are known, are characterised by their extremely high luminosities (close to the Eddington Limit, log (L/L_⊙) ~ 5.8), very high mass loss, late SpTs, and extreme SpT variations (see Humphreys & Ney 1974; Schuster et al. 2006; and see discussion on variability in Dorda et al. 2016). Finally, we have to note that the veiling effect is not constant. It increases and disappears along the spectral variation period of these stars (Humphreys & Ney 1974).

Because of the peculiarity and scarcity of RSGs affected by veiling, we propose here a few diagrams to identify them (see Figs. 14 and A.4). To locate the position of veiled RSGs, we have added to these diagrams our measurements of a spectrum of the ERSG S Per, which was observed with the INT, in the same conditions as the stars from the galactic sample. This star was not included in the PCA calculations precisely because of its well-known spectral variability.

In Fig. 14a, S Per (magenta star symbol) presents a sum of the CaT EWs of around 5Å, clearly lower than the giants. However, the strength of the TiO band is small, too small to weaken the CaT significantly in a supergiant. In fact, any giant with a similar TiO bandhead has a higher CaT value. Therefore it is isolated from any other star, except another RSG (with TiO bandhead of ~0.32 and CaT of ~5Å). This star is UY Sct, which was classified as M2–M4 Ia by Solf (1978). Even though its spectrum has not been reported as veiled before, UY Sct has been identified as the largest star known to date (Arroyo-Torres et al. 2013). Therefore, it seems only natural to count it as an ERSG.

These two ERSGs appear isolated in Fig. 14a. This is due to the nature of our sample, because we should expect mid- and late-M dwarfs (that we have not measured) to lie close to our ERSGs. As a consequence of veiling, our ERSGs are not identified as SGs by the PCA criteria developed above. The PC2 versus PC4 diagram (Fig. 14b) seems to be a good way to reveal the nature of these stars, as they lay among other RSGs. We have to caution, that we do not know where late-M dwarfs should fall in this diagram, even though the behaviour of other dwarfs suggests that they would occupy a different position. However, it is unlikely that red dwarfs and RSGs would be confused in the Galactic plane if IR photometry is available because the expected reddening is completely different in each case.

In summary, stars lying close to the position of our two ERSGs in the diagrams shown in Fig. 14 can be considered as candidates for veiled ERSGs. To confirm their nature, they should be observed throughout their spectral variation periods, because the veiling is expected to change along them. Thus, these stars would vary their position in the diagrams, and at some point they would be observed as normal (not veiled) RSGs.

5. Conclusions

In this work, we have developed criteria based on PCA and SVM methods to separate SGs from non-SGs through their spectral features in the CaT spectral region, using a statistically significant sample from the SMC, the LMC, and the Galaxy. We obtained an efficiency identifying SGs of 0.98 ± 0.04, and a contamination of 0.02 ± 0.04.

We also revisited those criteria used in the past to identify SGs (the sum of the EWs of the CaT, the blend at 8468Å, and the ratio between Fe i8514Å and Ti i8518Å), studying their behaviour for a significantly larger sample. We have evaluated their limitations and compared their efficiency and contamination with those obtained through our PCA/SVM method. We show that all classical methods present efficiencies similar to (strength of the CaT and the Fe i8514Å to Ti i8518Å ratio) or significantly lower (the blend at 8468Å) than the PCA method. However, their contaminations are significantly worse.

In conclusion, the PCA method is more reliable than the classical ones. Furthermore, as the PCA uses information from many different spectral features, it is more robust than those criteria which are based on a few lines.

The PCA method can also be applied to spectra taken with the RVS on board Gaia. These spectra cover a spectral region shorter than our spectra, but in Appendix A we repeat the same analysis, using only those lines that lie inside the RVS spectral range.

In spite of the good results obtained, we must highlight the fact that the efficiency and contamination depend on the typical SpTs in the sample (the mid- and late-M stars are hard to identify even with our PCA method), which depends on the metallicity of the population. We tested the PCA method for a sample from the Galaxy alone, obtaining a good efficiency (0.94 ± 0.13) and a low contamination (0.03 ± 0.13), statistically equivalent to the results from the three-galaxy sample.

In addition, we also investigated the behaviour of a number of features depending on SpT (TiO bandhead at 8859Å, the EW of the Ti lines, and the calculated PC2). From this we developed a method to estimate the SpT of CSGs.

Finally, we have also developed criteria to identify veiled RSGs, or at least good candidates for being such objects.

---

Acknowledgments

The observations have been partially supported by the OPTICON project (observing proposals 2010B/01, 2011A/014 and 2012A/015), which is funded by the European Commission under the Seventh Framework Programme (FP7). Part of the observations have been taken under service mode, (service proposal AO171) and the authors gratefully acknowledge the help of the AAO support astronomers. The INT and its service programme are operated on the island of La Palma by the Isaac Newton Group in the Spanish Observatorio del Roque de los Muchachos of the Instituto de Astrofísica de Canarias (observing proposals 024-INT2/11B and 97-INT8/12B, and the CAT service programme). This research is partially supported by the Spanish Ministerio de Economía y Competitividad (Mineco) under grants AYA2012-39364-C02-02, AYA2015-68012-C2-2-P (MINECO/FEDER), and FPI BES-2011-049345. The work reported in this publication has been partially supported by the European Science Foundation (ESF), in the framework of the GREAT Research Networking Programme. This research made use of the Simbad, Vizier, and Aladin services developed at the Centre de Données Astronomiques de Strasbourg, France. This publication makes use of data products from the Two Micron All Sky Survey, which is a joint project of the University of Massachusetts and the Infrared Processing and Analysis Center/California Institute of Technology, funded by the National Aeronautics and Space Administration and the National Science Foundation.

References

Appendix A: Application for Gaia

Appendix A.1: The Gaia spectral range

The spectral range covered by the Gaia spectrograph, RVS, is narrower than the range used in this work. Therefore, to apply our method to spectra from the RVS, we repeated the PCA calculations using only the lines inside its spectral range, from 8470Å to 8740Å.

We have built the Gaia input list by taking the shortened input list (see Sect. 3.1) and removing all the lines lying outside the Gaia spectral range. In addition, the lines Ti i 8734.5Å and Mg i 8736.0Å were not used either, even if they are inside the Gaia range, because their red continuum is beyond 8740Å and so their EW cannot be measured in Gaia spectra. In total, the Gaia input list contains three bandheads and 19 atomic lines.

Appendix A.2: PCA and SVM for the Gaia input list

We repeated the same PCA method described in Sects. 3.2.1 and 3.2.2, with the Gaia input list. Although it contains less features than our original list, the PCs obtained are similar to those presented before. The behaviour of the first three PCs is almost the same (see Figs. A.1 and A.2), and they together contain more than 90% of the accumulated variance. As has been explained before, the later PCs contain progressively less variance (see Fig. A.3), and so, to reach 98% of the accumulated variance of the Gaia PCA, the first nine PCs are needed. We used these nine PCs were for the subsequent SVM method. For the same reason, the standard deviations of the coefficients for PC8 and PC9 are significantly high.

We calculated the SVM boundaries following the same procedures presented in Sect 3.2.2. The coefficients obtained are given in Tables E.4–E.6.

Appendix A.2.1: Linear regressions

As the Gaia input list has given slightly different PC2 coefficients, we have recalculated the estimations of SpT through linear regression. In addition, one of the lines of the Ti index (Ti i8734.5Å) is not included in the Gaia input list. Thus, we have calculated a new Ti index for the Gaia input list, excluding this line. Finally, the TiO bandhead at 8859Å is outside the Gaia range, while all the other TiO bands inside this range are too weak to provide a clear relation for the whole M sequence. Therefore we have not used any TiO band to estimate the SpT.

The procedure has been similar to that outlined in Sect. 3.3, with some small differences. Firstly, the separation between SGs and non-SGs was done through the PCA method using the data for the M0 boundary, because it has the best efficiency and contamination in the identification of SGs in the case of the Gaia input list (see Sect. A.2.2). Secondly, in Sect. 3.3 we used the M1 type as the boundary between early and late subsamples, because the main TiO bandhead (at 8859Å) becomes noticeable at this subtype, changing the behaviour of the PC. This band is not inside the Gaia spectral range. Thus the PC2 behaviour does not change before M2, when the other lesser TiO bands start to be noticeable. In consequence we have used M2 as the boundary between early and late subsamples for these calculations. The coefficients obtained for the linear regressions are in Table A.1.

Appendix A.2.2: Efficiency

In Table. A.2 we compare the efficiency (i.e. the fraction of known SGs in the sample or subsample that are tagged as SGs by a given criterion) of the boundaries obtained for the Gaia input list in each boundary subtype for the Gaia input list.

With this spectral range, the efficiency is 0.97 ± 0.04 for all the five boundaries used, equivalent to that obtained for the complete line list in Sect. 4.1. The efficiencies for the early and late subsamples are statistically equivalent in all the cases, except for the boundaries M2 and M3, where we find significantly lower values. This is probably due to two factors. Firstly, the atomic lines less affected by the rise of TiO bands are those lying between 8700 and 8859 Å, and thus are not included in this analysis because they fall outside the Gaia spectral range. Secondly, the main TiO bandhead of the CaT region (at 8859Å) is also outside this window. On the other hand, the contamination is about 0.03 ± 0.04 for all the five boundaries, statistically equivalent to the result obtained for the complete line list in Sect. 4.1.

Appendix A.2.3: Veiled RSGs and the Gaia range.

Gaia spectra do not include the TiO bandhead at 8859Å. Therefore, we propose an alternative diagram to identify ERSGs

(see Fig. A.4), using PC2, because its behaviour is similar to that of the TiO bandhead. In the present case, the veiled RSGs occupy the same region as the early-M dwarfs in the sum of the CaT EWs versus PC2 diagram, but they may be identified as SGs by comparing their positions in the PC2 versus PC4 diagram (Fig. A.4b). Moreover, since Gaia will provide multi-epoch spectra, they may be used to check the different positions of a given star in these diagrams, as their veilings change along the spectral variation of these stars.

Appendix B: List of Milky Way stars

Appendix C: List of lines and bandheads measured

Appendix D: PCA coefficients calculated

Appendix E: SVM coefficients calculated

All Tables

All Figures