Free Access
Issue
A&A
Volume 576, April 2015
Article Number A78
Number of page(s) 17
Section Numerical methods and codes
DOI https://doi.org/10.1051/0004-6361/201423909
Published online 01 April 2015

© ESO, 2015

1. Introduction

Ground-based observations are naturally affected by various physical processes in the Earth’s atmosphere, in particular scattering and absorption. The dynamics of the weather conditions, seasonal effects, or climate change lead to variabilities in temperature, pressure, and the chemical composition on time scales from seconds to decades, making the absorption correction a demanding matter. Thus, any data calibration usually needs supplementary calibration frames. However, this approach is very expensive in terms of telescope time, because these data should be taken directly before or after the science target.

This applies particularly to the correction arising from molecular absorption in the Earth’s atmosphere. The required supplementary calibration frames are observations of telluric standard stars (TSS). The TSS are usually bright hot stars (e.g. B-type), that show only a few, well known intrinsic spectral features. The TSS have to be observed at a similar airmass, time, and line of sight as the science target to probe the same atmosphere column.

Within the framework of the in-kind contribution of Austria’s accession to the European Southern Observatory (ESO), we developed a comprehensive sky model1 covering a wavelength range from 0.3 to 30 μm (Noll et al. 2012). It was originally designed for the ESO Exposure Time Calculator2 and incorporates several components, such as airglow, scattered moonlight (Jones et al. 2013), zodiacal light (Leinert et al. 1998), scattered starlight, and the telescope emission modelled as a grey body. It also contains a spectral model of the Earth’s lower atmosphere in local thermal equilibrium calculated by means of the radiative transfer code LNFL/LBLRTM3 (Clough et al. 2005). This code is used with the spectral line parameter database aer, which is based on HITRAN4 (Rothman et al. 2009) and delivered together with the code. Averaged atmospheric profiles for Cerro Paranal describing the chemical composition as a function of height (combination of the ESO MeteoMonitor5, a standard atmosphere6 and the 3D Global Data Assimilation System (GDAS) model7) are also used as input. The GDAS model is provided by the National Oceanic and Atmospheric Administration (NOAA)7 and contains time-dependent profiles of the temperature, pressure, and humidity.

The ESO In-Kind software also includes tools for removing the atmospheric signature in spectra8,9. The package molecfit is an alternative approach for performing the telluric absorption correction by means of theoretically calculated transmission spectra based on the atmospheric model. We use the previously mentioned radiative transfer code and line database to derive synthetic transmission curves. The required atmospheric input profile is created in the same way as for the sky model, but for the time and location of the observations and not averaged. The algorithm and package functionalities are described in more detail in Smette et al. (2015, hereafter Paper I). Here, we evaluate the quality of molecfit as a telluric absorption correction tool for ESO Very Large Telescope (VLT) instruments. The focus lies on observations taken with the X-Shooter instrument (Vernet et al. 2011), an echelle spectrograph covering simultaneously the wavelength regime from the U to the K band at medium resolution. We use two methods: (a) a statistical study of the quality of the correction by means of figures of merit, and (b) a comparison of the molecfit correction with that obtained using the traditional method based on observations of TSS.

We first briefly describe the method of the telluric absorption correction incorporated in molecfit (Sect. 2). A description of the test data set is given in Sect. 3. In the following, we perform two different tests: in Sect. 4, we investigate the dependence of the telluric absorption correction quality on several parameters, i.e. the influence of the line transmission, observing conditions, spectral resolution, fitting ranges, input parameters, the atmospheric profiles, and the signal-to-noise ratio (S/N). For this purpose, we introduce two figures of merit to estimate the quality of the telluric correction Ioff and Ires, describing the systematic offsets and the remaining small-scale residuals of the corrections, respectively. These quantities are determined for a large data set of X-Shooter spectra, to which molecfit is applied with default parameters optimised for this instrument. The results of this analysis allow the user to estimate the achievable quality for the telluric absorption correction in case an individual parameter set optimisation is not possible, e.g. due to the large number of spectra. In Sect. 5, we compare the classical method based on a TSS with molecfit as another test. We have selected four science observations, for which both methods are optimised. In this way, individually optimised results can be compared for both methods. The results of this comparison are demonstrated by focusing on the scientifically relevant spectral details. Section 6 provides a summary of our findings.

2. Telluric correction with molecfit

The correction with the molecfit package is performed in two steps:

  • Determination of the best-fit atmospheric and instrumentalparameters: they are related to the total column densities of theinput atmospheric profile and the instrumental parameters(spectral line profile, wavelength calibration, continuumposition) are optimised by means of a χ2-Levenberg-Marquardt minimisation algorithm (see Paper I for a comprehensive description of the underlying algorithms) to best reproduce the observed telluric spectrum in selected wavelength regions. By varying the scaling factor of the molecular profiles of the initial input atmospheric profile, the programme iteratively calculates transmission curves, which are fitted to the input science spectrum. To minimise the calculation time, to optimise the continuum fit, and/or to avoid regions affected by stellar spectral features or instrument defects, the fitting is restricted to user-defined spectral ranges.

  • Correction of the telluric spectrum: the best-fit atmospheric and instrumental parameters are used to calculate the atmospheric transmission spectrum over the entire wavelength range of the scientific observation. The science spectrum is then divided by this transmission curve to produce telluric corrected data.

Table 1

Objects used for detailed tests.

3. The data set

In order to evaluate the performance of molecfit, we have used archival data obtained with the X-Shooter instrument mounted at the ESO VLT (Vernet et al. 2011). This instrument covers the entire wavelength range from 0.3 to 2.5 μm in three spectral arms (UVB arm from the ultraviolet to the B band; VIS arm in the visual regime; NIR arm in the near-infrared regime), at medium resolution (R ~ 3300 to 18 200, depending on the slit width) simultaneously. This broad wavelength regime gives the opportunity to study the correction of several absorption bands of different species simultaneously (e.g. H2O, O2, CO2).

Table 2

X-Shooter NIR-arm data set used for the detailed tests (see also Table 1).

We have reduced the entire publically available ESO archive data from October 2009 to March 2013, leading to a comprehensive data set taken under various observing conditions, since X-Shooter is frequently used. We used the ESO standard pipeline in version V2.0.0 under Reflex V2.3 on our cluster10.

For studying the influence of the atmospheric conditions and instrumental set-ups on the quality of telluric absorption correction (see Sect. 4), we have taken all 1D spectra of TSS without flux calibration until March 2013. In total, there are 4218 NIR-arm and 3823 VIS-arm spectra. UVB-arm spectra were not considered due to the lack of molecular absorption features11. Due to occasional failures of the automatic X-Shooter pipeline, the number of obtained spectra is lower than the number of exposures in the archive. Since the obtained set of reduced data was large enough for our purpose, we did not attempt to re-run the pipeline manually (possibly using tuned reduction parameters) for the cases for which the automatic approach failed.

Two observations of a γ-ray burst and a Seyfert 2 galaxy (#1 and 2 in Tables 1 and 2) were used to investigate the quality of the telluric absorption correction obtained with molecfit when applied to data with low S/N. Both observations show low count numbers (462 and 241 ADU, respectively) and were also taken at significant airmass, which increases the effect of atmospheric absorption.

For the comparison with the classical method described in Sect. 5, we have selected science observations of four different astrophysical objects (#3 through 6 in Tables 1 and 2) in conjunction with their corresponding TSS observations manually from our X-Shooter test data set in order to perform a comparison between the telluric absorption correction achieved with molecfit and the classical method related to TSS. The science test data sets were selected to cover different astrophysical objects and S/N traced by the counts in ADU (see Table 1) to also estimate the limits of the application of both methods. The TSS spectra were selected to coincide best with the airmass and the date of the corresponding science observation (see Table 2) and were usually reduced with the same flat field. As the majority of the telluric absorption features arise in the infrared regime, we restrict this investigation to NIR-arm data.

4. Quality of telluric absorption correction

In the following, we evaluate the quality of telluric absorption correction with molecfit by means of a large sample of X-Shooter TSS spectra (see Sect. 3). We mainly focus on the NIR arm, where the telluric absorption correction is most crucial. We also complement the discussion with results from the VIS arm. The molecfit test set-up for the data set and the quality indicators used are discussed in Sect. 4.1. The results are shown in Sects. 4.2 to 4.4. The effect of changing the fitting ranges and molecfit input parameters is discussed for an example spectrum in Sects. 4.5 and 4.6, respectively. The influence of differences in the input water vapour profile is described in Sect. 4.7. Finally, we investigate the effect of low S/N data on the fitting quality in Sect. 4.8.

4.1. Test set-up and figures of merit

Table 3

Molecfit parameter set-up for the telluric absorption correction evaluation of NIR-arm X-Shooter TSS spectra.

The performance of molecfit was tested with a fixed input parameter set. This approach is appropriate to estimate statistically the quality of the correction on a large set of data. Nevertheless, further quality improvements could be obtained by adjusting the fitting parameters for each individual spectrum.

For the NIR arm, the applied set-up is shown in Table 3 (see also Paper I). For the wavelength range from 1 to 2.5 μm, the model-relevant molecules are O2, CO2, H2O, CH4, and CO. For the VIS arm, it is sufficient to consider O2 and H2O. Only water vapour is fitted in spectra of both arms, since the concentration variations and the impact on the X-Shooter data of the other species are expected to be small. The equatorial standard atmospheric profile that we use is already more than a decade old (prepared by Remedios 2001; see Seifahrt et al. 2010). As the global CO2 concentration increases with time (World Meteorological Organization 2012), the input CO2 column was multiplied by 1.05 to be representative of the X-Shooter archival data.

A linear fit (cont_n = 1) was performed to correct the continua of the spectra in each of the fitting ranges (see below). The initial continuum factor cont_const was set to 104 to be in the order of the typical count level in ADU of X-Shooter data without flux calibration. For the wavelength grid correction, which is required to handle calibration uncertainties and an inaccurate centring of the target in the slit, only a constant shift was allowed (wlc_n = 0) (see Sect. 4.6 for a discussion). For the instrumental profile, a combination of a boxcar and Gaussian was assumed. A possible Lorentzian was not considered, since a study of the shape of the line profiles did not reveal significant Lorentzian wings. The width of the initial boxcar was chosen to be 75% of the slit width (relres_box), which should be close to the real value for the different slits (as confirmed by tests). An exception is the 5” slit, which is, however, rarely used for observations of science targets. For the Gaussian, a reasonable initial full width at half maximum (FWHM) of 1 pixel (res_gauss) and a kernel size of 3 FWHM (kernfac) were given. Since echelle spectra are fitted, the kernel FWHM was selected to be proportional to the wavelength (varkern = 1). The profile-related input parameters refer to the central wavelengths of 1.74 μm and 0.78 μm for the NIR and VIS arm, respectively. For the 214 spectra (5.1%) taken with a K-blocking filter (see Vernet et al. 2011), the corresponding wavelength is 1.55 μm.

Table 4

Wavelength ranges (vacuum) for fitting NIR- and VIS-arm X-Shooter TSS spectra.

thumbnail Fig. 1

Reference X-Shooter NIR-arm model transmission spectrum binned in 1 nm steps for a mean amount of precipitable water vapour (PWV) of 3.1 mm and an airmass of 1. The bins marked by red symbols are classified as continuum. The transmission of these bins ranges from 0.95 to 1.00. The mean value is 0.99. The selected bins are used to derive the systematic offsets (residuals) in the telluric absorption corrected spectra. Bins that belong to four specific transmission ranges corresponding to T = 0.9 (blue), 0.8 (cyan), 0.5 (magenta), and 0.2 (green) are indicated as well. The figure also shows the fitting ranges (yellow bars) that were used by molecfit (see also Table 4).

Open with DEXTER

thumbnail Fig. 2

Quality of telluric absorption correction for an example TSS spectrum taken with the NIR arm of the X-Shooter spectrograph. The star was observed with a 1.2′′ slit at an airmass of 1.32. The seeing was 0.76′′ and the PWV (as derived from the fitting) was 1.46 mm. Upper panel: telluric absorption corrected spectrum (red) is shown in comparison with the original spectrum (black). Middle and lower panels: mean counts and standard deviation in ADU for a grid of 1 nm bins for the telluric absorption corrected example spectrum. The two subfigures also highlight the pixels for the continuum interpolation (red) and the different transmission ranges for the quality analysis of the telluric absorption correction. For more details, see Fig. 1.

Open with DEXTER

As shown by Table 4 and Fig. 1, the fitting of the atmospheric transmission model in the NIR arm was restricted to five narrow 10 nm wide fitting ranges (or inclusion regions; cf. Paper I). They cover only about 3% of the entire wavelength range, which ensures that the fitting time is reasonable (typically 1 to 2 min) and the continuum fit with a low order polynomial is accurate enough. Moreover, only a relatively small fraction of the NIR-arm wavelength range is suitable for the fits, since a good fit requires a wide range of transmission values. The windows are sufficient to derive the amount of atmospheric water vapour (CO2 and CH4 are not fitted), the wavelength shift, and the instrumental profile. Since the last two properties have to be determined for the entire spectrum, Ranges 4 and 5, which do not show significant H2O absorption, are also important for a good coverage of the full wavelength range (see also Sects. 4.5 and 4.6). Moreover, our optimised set of fitting ranges avoids absorption features of typical TSS.

The VIS arm is much less affected by molecular absorption. The only prominent bands are the A and B molecular oxygen bands and the water vapour band at 0.94 μm. Therefore, fitting ranges were only defined in these three bands as indicated by Table 4. The H2O-related range avoids the wavelengths of potentially strong Paschen lines, which are present in spectra of hot TSS.

Running molecfit for the input parameters listed in Table 3 and the fitting ranges shown in Table 4 results in the best-fit parameters for each sample exposure (see Sect. 3). These are then used to provide a telluric absorption corrected spectrum for the full wavelength range (see Fig. 2). The quality of the fit in the pre-defined fitting ranges can be evaluated by considering the root-mean square (rms) of the residuals, which is provided by molecfit. However, the quality of the telluric absorption correction must be studied over the whole spectral range with respect to the quality of the corrected object spectrum and therefore requires a different analysis. For this purpose, we have defined the figures of merit Ioff and Ires. The former measures the continuum-normalised difference between the original and telluric absorption corrected spectrum, relative to a locally-averaged telluric absorption strength. The latter traces the continuum-normalised standard deviation of the residuals of the correction, relative to a locally-averaged telluric absorption strength. Therefore, Ioff and Ires are indicators of large-scale systematic offsets in the telluric absorption corrected spectra and small-scale (or high-frequency) variations of the residuals, respectively. These quality indicators were calculated in the following way:

  • First, each telluric absorption corrected NIR- and VIS-armspectrum was divided into 1 nm and0.5 nm wide bins, respectively (correspondingto about 17 and 25 pixels, i.e. a few resolutionelements; cf. Sect. 4.4).

  • The mean and the standard deviation were calculated for each of these bins (see Fig. 2).

  • The bins for which the average transmission model in Fig. 1 shows no or only minor absorption were selected as continuum nodes. Even though this selection includes weak outer wings of some bands, this is not critical. Lines with a depth of about 1% can be well corrected (see Fig. 2), especially with respect to the much stronger lines for which the quality of the correction is evaluated.

  • The continuum bins were used to interpolate the object continuum at the positions of bins with significant absorption.

  • The interpolated continuum intensity for each bin was then subtracted from the corresponding measured mean. This results in an estimate of systematic offsets in the telluric absorption corrected spectra.

  • To be independent of the absolute flux, the offsets and standard deviations of each bin were divided by the interpolated continuum mean values.

  • To make the resulting values independent of the wavelength- and time-dependent transmission T, the relative offsets and standard deviations were divided by 1 − T, i.e. the average amount of absorption for each bin (see Fig. 3).

  • Five transmission ranges were defined for a detailed analysis of the telluric absorption correction. Apart from a wide range from 0.1 to 0.95, ranges centred around 0.9, 0.8, 0.5, and 0.2 were defined (see Figs. 1 to 3). We did not consider T close to 1 and 0, since the related results are mainly noise driven and are therefore not suited to evaluate molecfit (see also Sects. 4.2 and 4.3).

  • The bins that belong to each of the five ranges were determined by using the best-fit transmission spectra of the individual sample spectra. Thus, the bin assignment depends on the airmass, amount of atmospheric water vapour, and spectral resolution.

  • Finally, to obtain the figures of merit and to reduce the effect of outliers, we took the median of the relative offset and standard deviation of the bins for each transmission range as shown in Fig. 3.

The resulting quality indicators Ioff and Ires are used in the subsequent analysis. If the transmission range is not specified explicitly, the results for the wide range are given. Note that Ioff and Ires have to be multiplied by 1 − T to provide the systematic offsets of the correction and the intra-bin standard deviations of the residuals relative to the object continuum. More details on the interpretation of our figures of merit can be found in Sect. 4.3.

thumbnail Fig. 3

Indicators Ioff for systematic offsets (upper panel) and Ires for small-scale residuals (lower panel) of the telluric absorption correction for single 1 nm bins of the X-Shooter NIR-arm spectrum shown in Fig. 2. Both quantities are given as a function of transmission. Bins belonging to the different transmission ranges used in the discussion are marked by different colours (see legend and Fig. 1). Small red symbols identify bins which are excluded from further analysis due to either very low transmission or very weak line strength.

Open with DEXTER

4.2. Outliers

thumbnail Fig. 4

Small-scale residual indicator Ires for 1 nm bins versus the mean model continuum counts in ADU for the five fitting ranges. The red dashed line shows the selection criterion for sufficiently high S/N for the detailed analysis.

Open with DEXTER

The indicator Ires measures variations in the count level within each of the narrow bins of a telluric absorption corrected spectrum. Primarily, this traces the small-scale quality of the telluric absorption correction. However, random noise, defects in the spectra, sky subtraction residuals, and spectral features of the observed object can also cause an increase of Ires. This is demonstrated for random noise in Fig. 4, which shows a clear increase of the scatter for lower mean counts (calculated for the fitted pixels), i.e. decreasing S/N. In order to avoid difficulties in interpreting the Ires sample statistics, spectra with mean counts less than 104 ADU are excluded from further analysis. This threshold concerns 120 spectra of the NIR arm (i.e. 2.8% of the sample) and 36 spectra of the VIS arm (0.9%). Excluding these data does not mean that their corrected spectra have a bad quality. For example, the mean Ioff for NIR-arm spectra with mean counts between 103 and 104 ADU is 0.000 (σ = 0.053), i.e. there are no systematic continuum offsets on average. For a discussion of the quality of the telluric absorption correction for low-S/N spectra, see Sect. 4.8.

The output files of the X-Shooter pipeline provide bad pixel masks, which can be used by molecfit to exclude critical pixels from the fitting procedure. Sometimes in the NIR arm, it seems that more pixels were rejected by the pipeline than required. In the case of a very small number of available pixels, the fit becomes unreliable. While the standard deviation might even decrease, systematic offsets are expected to become more significant. In addition, crucial continuum pixels for the interpolation of ranges with strong absorption bands could be missing, which makes the derivation of Ioff less reliable. For this reason, we excluded 247 NIR-arm spectra (5.9%) with less than half the maximum number of pixels in the fitting ranges.

So far, we have mainly rejected spectra where a proper calculation of the quality indicators Ioff and Ires could not be guaranteed. However, for evaluating the quality of the telluric absorption correction, it is important to know the fraction of obviously failed fits. For this purpose, we studied the best-fit FWHM of the instrumental profile (combined boxcar and Gaussian) and the best-fit shift of the wavelength grid. Interestingly, only a small number of NIR-arm spectra (no VIS-arm spectra) showed values which were clearly separated from the general distribution. 28 best-fit model spectra with a FWHM above 10 pixels (or below 1.5 pixels) or wavelength shifts of more than 2.5 pixels relative to the sample mean could be identified as clear outliers. 14 of these 28 spectra were already rejected by the critera described above. Molecfit appears to show a very robust performance, at least for X-Shooter TSS spectra.

For further analysis, we excluded all the discussed spectra with suspicious fits. This results in subsamples of 3837 NIR-arm (91% of the full sample) and 3787 VIS-arm spectra (99%).

4.3. Influence of line transmission and observing conditions

Table 5

Sample averages and standard deviations for the transmission-dependent indicators Ioff and Ires of the quality of the telluric absorption correction of NIR-arm X-Shooter spectra.

thumbnail Fig. 5

Indicators Ioff for systematic offsets and Ires for small-scale residuals of the telluric absorption correction for 1 nm bins. Only data points of the filtered subsample are shown.

Open with DEXTER

For the filtered NIR-arm sample (see Sect. 4.2), Table 5 shows the mean values and standard deviations of Ioff and Ires for the five transmission ranges listed. In addition, the individual values for the wide transmission range from 0.1 to 0.95 are plotted in Fig. 5. The latter does not indicate significant features in the distribution of the data points. Ioff clusters around a value of 0, which means that the telluric absorption correction does not appear to be affected by systematic offsets. The scatter is only about 3% of the line strengths. The mean Ires is about 0.20 with a scatter of 0.06, i.e. the relative standard deviation of the residuals of the corrected telluric absorption is about 20%. This translates into typical errors of 2% and 10% relative to the continuum for spectral ranges with T = 0.9 and 0.5, respectively. The mean Ioff and Ires values for the X-Shooter VIS arm −0.002 (σ = 0.048) and 0.184 (σ = 0.059) are very similar to the NIR-arm results. This implies that the telluric absorption correction is of good quality in both X-Shooter arms. The results are consistent with a typical correction accuracy of 2% of the continuum or better for wavelength ranges with unsaturated telluric lines as reported in Paper I based on data from different instruments. Note that individual molecular lines are not resolved in the X-Shooter spectra, which lowers and smoothes the measured telluric absorption.

Table 5 lists Ioff and Ires depending on the four narrow transmission ranges centred at T = 0.9, 0.8, 0.5, and 0.2. The distributions agree quite well with the results for the wide transmission range. Consequently, the mixing of transmissions is not crucial for the resulting quality indicators, at least if T very close to 1 and 0 are not considered (as for our wide T range). In the case of very high T, the figures of merit are no longer reliable with respect to the quality of the telluric absorption correction, since random noise, systematic errors in the reduced spectra, and features of the observed object can have a strong effect due to the normalisation by 1 − T (see Fig. 3). The correction quality is probably comparable with the results for intermediate T, which are crucial for the model fit for all T. In the case of very low T, the relatively low S/N, possible zeropoint errors, and the strong variation in T over narrow wavelength ranges can cause high Ioff and Ires. The correction of spectral ranges with T close to 0 is difficult (see Fig. 2). However, this is usually not an issue, since the information from the science target also tends to be very limited. The Ires values in Table 5 illustrate the described effects. The minimum of 0.15 is obtained for intermediate transmissions (T = 0.5), whereas the values for T = 0.9 and 0.2 are above 0.2.

thumbnail Fig. 6

Fraction of NIR-arm bins with standard deviations of the residuals of the telluric absorption correction greater than or equal to 5% of the object continuum as function of the best-fit PWV in mm. The red regression line has a slope of 0.15 per dex.

Open with DEXTER

The rough proportionality of the telluric absorption correction errors relative to the continuum and 1 − T for a wide range of T implies that the overall correction quality of a spectrum is correlated with properties which affect T, i.e. the airmass of the observation and the column density of the molecular species concerned. This can be understood by considering that the transmission T is related to the optical depth along the line of sight τ for each wavelength λ by (1)where τ can be approximated by the product of the optical depth at zenith τ0 and the airmass X: (2)This works best if the geometrical distributions of the given molecule and air are similar. Finally, the optical depth τ0 for molecular absorption by a single species at wavelength λ can be calculated by (3)where σabs is the wavelength-dependent absorption cross section and the integral corresponds to the column density of the molecule, which is derived from the density n at heights h above the observer at h0. Consequently, telluric absorption correction is most difficult if a target is observed at a large zenith distance and with a high atmospheric water vapour content. The amount of water vapour is critical, since most prominent bands in the X-Shooter wavelength range are caused by this molecule and the concentration and distribution is highly variable in time and space.

The effect of water vapour on the quality of the telluric absorption correction is shown in Fig. 6, which displays the fraction of 1 nm NIR-arm bins with residuals greater than or equal to 5% of the corrected object continuum as function of the amount of precipitable water vapour (PWV) in mm as derived by molecfit. For our X-Shooter NIR-arm TSS sample, the fitted PWV range from 0.2 to 18.2 mm with a mean value of 3.1 mm. The median is 2.2 mm. These values are in good agreement with other measurements of the PWV, which is routinely monitored at Cerro Paranal by a stand-alone microwave radiometer in support of science observations (see Kerber et al. 2014, and Sect. 4.7). There is a clear, nearly linear increase of the number of bins with significant residuals with increasing PWV in logarithmic units. A regression analysis indicates that the fraction fσ ≥ 0.05 grows from about 0.20 at 1 mm to about 0.35 at 10 mm. For comparison, the fraction of bins with a transmission lower than 0.95 of the mean spectrum plotted in Fig. 1 is about 0.5. The effect of the airmass on fσ ≥ 0.05 is smaller than for the PWV, since the airmass only varies by a factor of about 2. In the VIS-arm range, the regression line exhibits an increase of fσ ≥ 0.05 from about 0.04 at 1 mm to 0.07 at 10 mm. Since strong (water vapour) lines are rare in this wavelength regime, the fractions are distinctly smaller than in the NIR-arm range.

thumbnail Fig. 7

Best-fit PWV obtained by molecfit in mm as function of the observing date in years.

Open with DEXTER

The quality of the correction of water vapour bands may roughly depend on the time of the year. Figure 7 indicates a strong seasonal dependence of the atmospheric water vapour abundance. In winter, the PWV is relatively low (mean of 1.9 mm for meteorological winter), whereas the highest amounts and a large scatter are found in summer (mean of 5.2 mm). The strength of molecular oxygen and carbon dioxide bands can be considered as nearly stable except for the long-term increase of the CO2 concentration (see Sect. 4.1).

4.4. Influence of resolution

Table 6

Slit-dependent quality of the telluric absorption correction of NIR-arm X-Shooter spectra.

thumbnail Fig. 8

Small-scale residual indicator Ires for 1 nm bins versus the FWHM of the line profile in pixels. As indicated by the legend, the different colours correspond to different slit widths.

Open with DEXTER

Since the echelle gratings of the X-Shooter spectrographs are fixed, the slit width is the only instrumental parameter that affects the line-spread functions of the resulting spectra. For the NIR and VIS arms, seven different widths from 0.4′′ to 5.0′′ can be selected as displayed in Table 612. Apart from the slit selection, the positioning accuracy of the target in the slit and its change with time contribute to the resulting line-spread function. Finally, the FWHM of a line is influenced by the seeing at the time of the observation of the point-like standard star, especially if the slit is larger than the light profile of the target. For this reason, Table 6 shows an increase of the FWHM (as derived from the combined best-fit boxcar and Gaussian kernels, see Sect. 4.1) as well as its scatter with increasing slit width for the NIR arm.

The slit-dependent results for the quality indicators Ioff and Ires for the wide transmission range are also provided by Table 6. The systematic offsets appear to indicate a weak trend from slight overcorrection for the 0.4′′ slit to moderate undercorrection for the 5.0′′ slit. However, except for the value for the rarely used 5.0” slit, the mean offsets can be considered as negligible. Nevertheless, the slit width seems to contribute to a broadening of the Ioff distribution of the entire data set (see Fig. 5 and Table 5).

As indicated by Fig. 8 and Table 6, the intra-bin variations of the residuals of the telluric absorption correction increase with decreasing slit width or FWHM. The mean Ires values range from 0.156 for the 5.0′′ slit to 0.239 for the 0.4′′ slit. For the VIS arm, the corresponding values are 0.153 and 0.219. This clear dependence broadens the distribution of Ires for the entire data set (see Fig. 5 and Table 5). At first, the trend can be explained by the expected steepness of the line profiles if a line comprises only a few pixels (about 2.5 pixels for the 0.4′′ slit of the NIR arm). In this case, small discrepancies between the modelled and the true profile can cause significant residuals. On the other hand, the scatter in the NIR/VIS arm is calculated for bins of a width of 1 nm/0.5 nm (about 17/25 pixels). If the FWHM is low, more (probably uncorrelated) resolution elements fit into the bin range. This effect could also augment Ires.

4.5. Influence of fitting ranges

So far, the discussion has been based on a fixed set of fitting ranges (see Table 4 and Sect. 4.1). In the following, we focus on changes in the quality of the telluric absorption correction when these ranges are changed. For this purpose, we tested the NIR-arm example spectrum shown in Fig. 2, which is characterised by an airmass of 1.32 and a best-fit PWV of 1.46 mm. We ran molecfit for different subsets of the five standard NIR-arm fitting ranges, which had to include at least one range dominated by water vapour lines.

Table 7

Influence of fitting ranges on the telluric absorption correction of the NIR-arm spectrum displayed in Fig. 2.

Table 7 shows the results of the seven test runs we performed. The table indicates the selected fitting ranges (see Table 4 for the numbers), the rms of the fit residuals relative to the mean counts, the best-fit FWHM, the best-fit PWV, and the two quality indicators Ioff and Ires (see Sect. 4.1). The table entries for the different parameters show a clear difference in the quality of the fit and the telluric absorption correction depending on the fitting ranges considered. As expected, the rms of the fit is reduced if the number of the fitting ranges is decreased. The resulting FWHM range from 69% to 101% of the value of the standard run. For the PWV, we obtained values from 90% to 115%. The largest deviations are found for runs that were based on only one fitting range. For the quality of the telluric absorption correction as measured by Ioff and Ires, there is a similar trend. However, the quality of the correction also strongly depends on the ranges involved. While the fit only depending on Range 1 (1.12 to 1.13μm) is by far the worst (Run 2), the result for Run 4, which is only based on Range 3 (1.80 to 1.81μm), is remarkably good. This is also illustrated by Fig. 9, which shows the mean values and relative standard deviations for 1 nm bins of the resulting spectra of Run 2 and 4 in comparison with the standard Run 1. Run 2 only led to a good telluric absorption correction in the fitted range, whereas Run 4 shows a reasonable correction over the entire wavelength range.

Since the results for the molecfit runs with single fitting ranges differ significantly, we performed the model fitting and telluric absorption correction for single ranges that cover telluric lines in the same bands such as Runs 2 and 3, but with different wavelength limits. Only ranges where lines of intermediate strength dominate were selected (see Fig. 1). For ranges within the same H2O band, Table 8 reveals similar values for the listed parameters. This suggests that changing centre and width of a fitting range in a band (in a reasonable way) has less of an impact than changing the band.

thumbnail Fig. 9

Mean counts in ADU and standard deviation relative to mean counts for a grid of 1 nm bins of the telluric absorption corrected example spectrum shown in Fig. 2 for three different sets of fitting ranges. The black spectrum equals the one in Fig. 2 and corresponds to the standard set-up of windows described in Table 4. The red and the green spectra were calculated by only using a single fitting range (see legend and Table 7).

Open with DEXTER

Table 8

Influence of changing fitting ranges in a molecular band on the telluric absorption correction of the NIR-arm spectrum displayed in Fig. 2.

These results imply that modifying the fitting range within the H2O band at 1.13 μm does not significantly improve the quality of the fit. The PWV, Ioff, and Ires values remain unsatisfying if a fitting range within this band is not combined with ranges in other molecular bands. Since the line depths of the different ranges are comparable, this does not seem to explain the discrepancies. Differences in the best-fit line shapes and wavelength shifts (up to 1 pixel) for the different molecfit runs could imply that the issue is linked to the structure of the X-Shooter composite echelle spectra consisting of many orders. At least, line profiles in the 1.13 μm band cover less pixels than those in bands at longer wavelengths (see Sect. 4.4). In view of the uncertainties in the line profile and wavelength calibration, a good telluric absorption correction over the entire wavelength range (see Fig. 9) requires that all critical molecular absorption bands are probed by fitting ranges. Therefore, our standard set of fitting windows (see Table 4) is well defined, even though only low order corrections of the different kinds of systematic deviations from the atmospheric transmission model are possible (see Sect. 4.6). The best fit is always the result of a compromise, as indicated by the smaller residuals for individual fitting ranges in the corresponding wavelength regimes (see Fig. 9).

4.6. Influence of input parameters

Table 9

Influence of wavelength grid correction by a Chebyshev polynomial of order wlc_n on the telluric absorption correction of the spectrum displayed in Fig. 2.

Table 10

Influence of wavelength grid correction by a Chebyshev polynomial of order wlc_n on the telluric absorption correction of the spectrum displayed in Fig. 2 for the fixed best-fit line profile (relres_box = 0.560 and res_gauss = 1.671) as derived from the standard test run.

thumbnail Fig. 10

Mean counts in ADU and standard deviation relative to mean counts for a grid of 1 nm bins of the telluric absorption corrected example spectrum shown in Fig. 2 for different orders of the Chebyshev polynomial for the wavelength grid correction and fixed line profile (see legend and Table 10).

Open with DEXTER

Next, we investigate the influence of the input fit parameters on the quality of the telluric absorption correction. In this respect, our input parameter set (see Table 3) appears to be a reasonable choice (see discussion in Sect. 4.1). However, there could be a critical restriction of the maximum order of the Chebyshev polynomial for the correction of the wavelength grid. The selection wlc_n = 0 only allows a constant shift of the wavelength grid. As discussed in Sect. 4.5, this is most likely not sufficient to achieve good fits in all the different fitting ranges at the same time. For this reason, we studied the effect of the degree of the Chebyshev polynomial on the quality of the fit. Table 9 shows the results of our investigation of the standard NIR-arm example (see Fig. 2) for four different degrees of the polynomial. The result columns are the same as in Table 7. The values for the relative rms, FWHM, Ioff, and Ires clearly imply that the fits of the runs with wlc_n = 1, 3, and 5 failed. The doubling of the FWHM for these runs indicates an increase of the degrees of freedom by the additional coefficients of the Chebyshev polynomial caused degeneracies, which led to an erroneous fit of the instrumental profile. In other words, the fitting algorithm was not able to find the global χ2 minimum.

To make the wavelength correction more robust, we performed a second series of runs with reduced degrees of freedom. For this purpose, we fixed the properties of the line profile. We took the best-fit results of the standard run and set relres_box = 0.560, res_gauss = 1.671, and the corresponding fit flags to 0 (cf. Table 3). The results for the four different degrees of the Chebyshev polynomial are listed in Table 10. For the higher wlc_n, the fits in the five fitting ranges are now better than for the standard run, as the relative rms indicate. The PWV values and the related Ioff for systematic offsets are relatively stable. The small-scale residuals indicator Ires is the lowest (0.195) for a linear wavelength correction function, i.e. wlc_n = 1. Higher order corrections indicate worse Ires (0.343 for wlc_n = 5). They tend to deteroriate the telluric absorption correction, despite the fixed line profile. This is also demonstrated by Fig. 10, which shows the mean values and relative standard deviations for 1 nm bins of the telluric absorption corrected spectra for the different runs. For higher degrees of the polynomial, the quality of the correction seems to be highly wavelength dependent.

These findings suggest that very high wlc_n are risky because of fit degeneracies. The situation could improve if there were more and/or broader fitting ranges. However, for the wavelength range covered by X-Shooter, this approach is not feasible due to the small fraction of wavelengths with suitably strong absorption lines. Irrespective of these issues, the example has shown that a combination of two runs (where the second run benefits from the results of the first run) could significantly improve the quality of the telluric absorption correction.

For a successful χ2 minimisation, the number of free parameters should not be too high. Apart from the wavelength grid correction, the line shape parameters are prone to χ2 degeneracies. Line blends by low spectral resolution and only small numbers of telluric absorption lines by very narrow fitting ranges can be critical. Fortunately, the X-Shooter spectra do not seem to require modelling of possible broad line wings by a Lorentzian kernel component (see Table 3), which makes the fits more robust.

4.7. Influence of the input atmospheric profile

thumbnail Fig. 11

PWV in mm versus the best-fit scaling factor for the input water vapour profile (output value for relcol parameter; see Table 3).

Open with DEXTER

Water vapour is an abundant and very variable component of the Earth’s atmosphere (see Fig. 7). Most telluric absorption in the X-Shooter NIR-arm range is caused by this molecule. Therefore, the quality of the telluric absorption correction strongly depends on a good fit of the water vapour column density.

The PWV fit corresponds to a scaling of the input water vapour profile. The parameter relcol describes the relative scaling with respect to the input profile. Figure 11 shows the final PWV value from the best fit versus the best-fit relcol (see Table 3) for the NIR-arm data set selected in Sect. 4.2. The mean factor is 0.87 with a scatter of 0.26. This is relatively close to 1, i.e. the case that the PWV of the input profile is the best-fit one. However, for low PWV, the merged input profiles tend to have too much water vapour. For PWV below the median value of 2.2, the mean scaling factor is 0.73, whereas for PWV above the median, a mean factor of 1.01 is obtained. The standard deviation is similar in both cases (0.22 versus 0.23). In view of the significant scatter and the systematic offsets at low PWV, a reliable scaling of the input profiles is indispensable.

To more accurately investigate the effect of the initial atmospheric profiles, we used data of a radiometer, which was installed on Cerro Paranal in October 2011 for water vapour monitoring purposes (Kerber et al. 2012a,b). It is a Low Humidity And Temperature PROfiling microwave radiometer (LHATPRO), manufactured by Radiometer Physics GmbH (RPG13). The instrument uses several channels across the strong water vapour emission line at 183 GHz, necessary for measuring the low levels of PWV that are common on Cerro Paranal. Details of the radiometer are described in Rose et al. (2005). This radiometer provides continuous direct on-site measurements of the temperature and water vapour content. It also calculates the pressure profile up to a height of about 12 km above Cerro Paranal, and the integrated water vapour (IWV), identical to the PWV. We created an additional set of initial atmospheric profiles for molecfit in the same way as the default combination of a standard atmosphere, GDAS model, and ESO MeteoMonitor described in Paper I, but replaced the GDAS by the radiometer profiles and skipped the MeteoMonitor data since the latter profiles already contain this information.

From our X-Shooter data set, we selected 549 TSS observations obtained in Jan., Feb., and Aug.-Dec. 2012, which correspond to the period of the radiometer data provided to us by ESO. Every spectrum was fitted with molecfit incorporating both, the GDAS/MeteoMonitor and the LHATPRO based set, respectively. In both cases, we used the closest available profiles. We finally applied a telluric absorption correction based on both methods.

The telluric corrected spectra differ usually by only a few per cent, with larger deviations in the ranges affected by strong atmospheric absorption (see Fig. 12 for an example).

For a closer look, we also compared the resulting relcol values and the water vapour content values, PWV and IWV respectively, for the whole data set. Figure 13 gives the comparison of the relcol parameters between the GDAS and the LHATPRO based fits. As expected, the LHATPRO based scaling parameters are closer to unity (median value = 0.95) than the relcol parameter derived with the help of the GDAS model (median value = 0.92). Also the relcol scatter σH = 0.13 for the LHATPRO method is significantly lower than the GDAS based scatter (σG = 0.25). This means that the atmospheric profile based on the radiometer data requires less scaling than the modelled one. This is expected, since the LHATPRO profiles are direct on-site measurements providing more accurate estimates of the actual atmospheric conditions than the combined GDAS/MeteoMonitor model.

Although the radiometer based initial profiles lead to less scatter in the relcol parameter, the quality of the final telluric absorption correction is comparable within a few per cent. This means that due to the adaption with the scaling parameter, an inappropriate initial atmospheric profile also leads to a good telluric absorption correction. We therefore conclude that the underlying fitting algorithm incorporated in molecfit is highly efficient. However, it appears that the derived PWV value is overestimating the real water vapour content in the case of very dry observing conditions. This indicates inadequate input profiles.

thumbnail Fig. 12

Relative difference of the telluric absorption corrected spectra of the TSS Hip039634 (IWV = 1.5 mm), one corrected with a LHATPRO, and one with a GDAS based profile (in per cent). ctsH = counts of the LHATPRO based telluric absorption corrected spectrum; ctsG = GDAS counterpart.

Open with DEXTER

thumbnail Fig. 13

Comparison of the relative water vapour column scaling factors relcol derived with the standard GDAS/MeteoMonitor model and the LHATPRO measurements for 549 TSS spectra.

Open with DEXTER

4.8. Influence of the S/N

In this section, we investigate the performance of molecfit with respect to low signal data. For this purpose, we used a spectrum of the γ-ray burst GRB 130606A (Xu et al. 2013; Ukwatta et al. 2013; Castro-Tirado et al. 2013; de Ugarte Postigo et al. 2014) at redshift z ~ 5.913 (Hartoog et al. 2014) taken in June 2013 (Prog.-ID: 091.C-0934; P.I.: Kaper). Figure 14 shows the uncorrected (upper panel), the transmission (middle panel), and the corrected spectrum (lower panel). Although the object flux has a low mean count level of 241 ADU, molecfit was able to reliably correct the telluric absorption in wide parts.

Another low S/N example is the spectrum of PKS1934-63, a Seyfert 2 galaxy, taken in July 2011 (Prog.-ID: 087.B-0614; P.I.: Holt). These data contain a comparably low median count level of only 462 ADU (cf. Table 1). Figure 15 shows as an example the three wavelength ranges 1.1 to 1.3 μm (upper panel), 1.3 to 1.5 μm (middle panel) and 1.9 to 2.1 μm (lower panel), respectively. The first two ranges covers the strong water vapour absorption bands between 1.1 and 1.17 μm and 1.34 and 1.52 μm, respectively. The correction for minor absorption regions is reasonable. The prominent CO2 features between 2.0 to 2.08 μm plotted in the lower panel are corrected well.

However, strong absorption bands cause major problems when correcting low ADU data. Figure 15 also shows the strong absorption bands between 1.34 to 1.52 μm (panel a) and 1.8 to 1.94 μm (panel b) arising from water vapour and carbon dioxide, respectively. In these regions, the correction leads to large uncertainties, since the incorporated division and the low signal level increase the noise significantly. We therefore conclude that the telluric absorption correction with molecfit can be critical in such cases. For very low S/N data it might be even impossible to apply molecfit. In this case, the user might consider observing TSS to derive the correction function.

thumbnail Fig. 14

X-Shooter NIR-arm spectrum of γ-ray burst GRB 130606A at z = 5.913. The graphs show the wavelength range from 1.15 to 1.16 μm, which is affected by both telluric and some intrinsic absorption lines arising at a redshift z = 3.4515 (Hartoog et al. 2014). Upper panel: uncorrected spectrum. Middle panel: transmission spectrum as calculated by molecfit. Lower panel: corrected spectrum.

Open with DEXTER

thumbnail Fig. 15

X-Shooter NIR-arm spectrum of the Seyfert 2 galaxy PKS1934-63 (z ~ 0.183). Upper panels of a), b), and c): uncorrected and the molecfit corrected spectrum in different wavelength ranges. This observation has a low mean ADU level of 462 counts. The displayed transmission curve (lower panels) is based on a fit in the standard fitting regions (yellow areas).

Open with DEXTER

5. Comparison with the classical method

5.1. Method

The classical way of the telluric absorption correction is done with the help of TSS, which are used to derive the transmission of the Earth’s atmosphere at the time of the observation. Since TSS are stars with few or well known spectral features, they can be used to obtain an atmospheric transmission after subtracting their continuum and subsequent normalisation. We used the following approach: we fitted a cubic spline to base points selected on positions with or close to transmission T = 1 in the TSS spectrum in order to determine the continuum. This spline fit is then used to normalise the TSS spectrum resulting in a transmission spectrum (see Fig. 16). We corrected the science spectrum with this transmission using the IRAF task telluric14. This task also performs a wavelength shift and a scaling of the input spectrum to achieve the telluric absorption correction.

For the comparison of the quality of the telluric absorption correction between classical TSS and molecfit methods, we used X-Shooter NIR-arm spectra of four different scientific objects, a B(e) star, an E0 galaxy, a Carbon star, and a planetary nebula (PN), in conjunction with their corresponding TSSs (see Tables 1 and 2). We divided the wavelength range covered by the NIR arm into 16 pieces: eight regions with major atmospheric absorption features (labelled with the numbers #1 through #8) and another eight regions with minor absorption (labelled with the letters “a” through “h”, see Fig. 16 and Table 11). To optimise the telluric absorption correction, an individual set of fitting parameters was derived for each spectral region with high absorption. Ranges with minor atmospheric absorption were corrected with molecfit using the standard parameters given in Table 3. To achieve a comparison based on optimised conditions for both methods, the classical method was also applied to each spectral piece individually. In addition, the positions of the base points for the continuum fit were individually chosen for every TSS spectrum to obtain an optimal transmission curve.

5.2. Results

The comparison of the quality of the telluric absorption correction achieved with the classical TSS and molecfit method reveals that in some regions with minor atmospheric absorption both methods perform similarly, e.g. the weak CO2 bands between 1.6 and 1.615 μm are well corrected (see Fig. 17). There are also some regions with major atmospheric absorption, where both methods achieve very good absorption correction (e.g. redwards of about 2 μm in Fig. 18).

However, usually noticeable differences in the quality of the correction are visible. These differences even become critical for regions #1 through #8, where major absorption bands affect any ground based observation. In total, we identified three classes of critical problems.

Table 11

Individual regions used for the telluric absorption correction.

thumbnail Fig. 16

Upper panel: telluric standard star spectrum Hip084409 (test data set #3, see Tables 1 and 2) and the fitting method for extracting a transmission curve: a cubic spline fit to base points (blue dots in upper panel) is used to derive the continuum of the telluric standard star. This fit is then used to normalise the spectrum to achieve the corresponding transmission (lower panel). We divided the spectrum into several wavelength regions based on the amount of absorption. Regions #1 through #8 are heavily affected by absorption, in contrast to regions (a) through (h) (see Table 11). Each region was corrected individually with molecfit and IRAF. Middle panel: resulting transmission curve including some molecular absorption features. Lower panel: theoretical transmission curve achieved with molecfit.

Open with DEXTER

5.2.1. Object continuum reconstruction

Reconstructing the continuum of the science target in a reliable way is a difficult issue, particularly in regions with broad absorption regions like #3 and #4 (see Fig. 16 and Table 11). The quality for the continuum reconstruction with this implementation of the classical method crucially depends on the incorporated interpolation. As such, a fit only can be based on a limited number of base points. Artificial continuum variations are unavoidably introduced in any wavelength range, even with minor or no molecular absorption. Figure 19 shows an example of a poor continuum reconstruction in a minor absorption region, where the Brackett series and some FeII lines of the B(e) star (Kraus et al. 2012) arise (see Fig. 19a). The continuum is not well reproduced in the region around the Brackett line (411), even though this region is only marginally affected by absorption. Another example is shown in Fig. 18, where the spectrum of the E0 galaxy NGC 5812 is given in the range of a prominent CO2 absorption region. The continuum shortwards of about 2 μm derived with the TSS method is too low, leading to an overcorrection of the continuum.

5.2.2. Line correction/reconstruction

Reliable reconstruction of object lines can be a difficult matter for the telluric absorption correction. It becomes critical if a line intrinsic to the TSS affects a scientifically important line in the science spectrum. For example, this applies to hydrogen lines, which is demonstrated by a spectrum of the PN IC1266.

thumbnail Fig. 17

Comparison of the telluric absorption correction methods: uncorrected object spectrum (black), molecfit corrected spectrum (red line), and the spectrum corrected with the classical method (green) in a region with minor atmospheric absorption (CO2 band).

Open with DEXTER

thumbnail Fig. 18

Upper panel: example for poor continuum reproduction by the classical method (colour coding as in Fig. 17). Although the prominent CO2 absorption feature is well corrected by both methods, significant variations in the continuum are visible in the spectrum corrected by the classical method. Intrinsic object lines redwards of 2 μm are well reconstructed by both methods. Lower panel: relative residuals of the corrected spectra when object lines are excluded from the fit in region #5 (blue areas in upper panel).

Open with DEXTER

Figure 20 shows the hydrogen Paβ line at 1.2818 μm, which has significantly higher flux and much broader wings when corrected with the classical method. This is induced by the fitting method to derive the transmission curve, which shows significant differences compared with the one derived with molecfit (see lower panel of Fig. 20). In addition, a small wavelength shift is visible in Fig. 20 that is probably caused by a small radial velocity shift affecting the TSS Hip085885.

Another example is given in Fig. 21, which shows a spectrum of the E0 galaxy NGC 5812 with significant overcorrection by the classical method arising from TSS Brackett lines. As molecfit incorporates a purely theoretical transmission curve, such intrinsic stellar features do not occur.

One of the possible drawbacks of using molecfit could be the sensitivity of the fit to intrinsic object lines, if they have not been excluded. To investigate this, we used the spectrum of data set #4, the elliptical galaxy NGC 5812, which shows some intrinsic features in region #5 between 1.995 and 2.035 μm (see upper panel of Fig. 18). This region is dominated by carbon dioxide and minor water vapour absorption. We only used the fitting range #5 (between 1.9918 and 2.0333 μm), where H2O and CO2 were varied. It can be seen that the intrinsic object features remain unchanged (see Fig. 18). To estimate how sensitive molecfit is with respect to object lines, we again fitted region #5, but excluded the object lines from the fit (see blue areas in upper panel of Fig. 18). We find only marginal differences in the telluric absorption corrected spectra of ± 20 ppm. We therefore consider molecfit to be robust with respect to single object lines, at least if object features do not dominate the corresponding fitting range. However, care should be taken when prominent intrinsic spectral object features or even entire bands are expected at the same wavelength as strong telluric absorption lines. These features might be visible e.g. in low mass stars, molecular clouds, or planetary atmospheres. Prominent features indeed might influence the fit by mimicking differing molecular abundances in the Earth’s atmosphere. Hence, the contributions of the astronomical target and the Earth’s atmosphere cannot be disentangled by molecfit. In this case, the user is advised to either mask these features, if applicable, or choose different fitting ranges to avoid an unintended removal of object features. Alternatively, if such features are known to be visible in the entire spectrum, molecfit can be applied to a corresponding TSS and subsequent usage of the resulting transmission spectrum for the correction of the science target.

thumbnail Fig. 19

Panel a): Brackett series from V921-Sco, a B(e) star. Panel b): zoomed in region with bad continuum correction by the classical method (colour coding as in Fig. 17).

Open with DEXTER

thumbnail Fig. 20

Paβ line visible in the spectrum of the PN IC1266. Upper panel: although the line is located in a wavelength range with minor atmospheric absorption, it is not well corrected with the classical method. An intrinsic wavelength shift is probably introduced by the radial velocity of the object (colour coding as in Fig. 17). Lower panel: transmission derived with the classical method (green line) shows a broad dip leading to a significant higher flux and different line shape in the corrected spectrum. The transmission derived with molecfit (red) does not show this feature.

Open with DEXTER

5.2.3. Increased noise

The finite S/N of the TSS becomes critical when it is smaller or comparable to the one of the object spectrum. Applying a noisy transmission for the telluric correction derived by means of such a spectrum unavoidably degrades the S/N of the science spectrum. Figure 22 shows a spectral region with negligible atmospheric absorption, where both methods usually lead to a good reproduction of intrinsic spectral object features (except the variations at 1.094 μm introduced by the TSS transmission). Since the theoretical transmission curve obtained with the radiative transfer code does not show any random noise, molecfit does not introduce noise-driven features in the science spectrum. However, the classical method does change the noise level. This becomes more and more significant, the more prominent the atmospheric absorption features are. For example, the prominent bands from 1.3 to 1.5 μm (H2O) and from 1.8 to 2.0 μm (H2O and CO2) lead to very noisy science spectra, when the telluric absorption correction is done with the classical method (see Figs. 23 and 24, respectively). This is particularly important when object features of scientific interest are located there.

thumbnail Fig. 21

Example of an overcorrection by spectral features arising from the Brackett lines (H(410), H(411), and H(412)) visible in the TSS. In the molecfit based correction such features do not occur (colour coding as in Fig. 17).

Open with DEXTER

thumbnail Fig. 22

Upper panel: detailed zoomed in region with minor atmospheric absorption. Due to variations caused by the spline fit and noise in the TSS spectrum, the classical method introduces continuum variations and noise to the corrected spectrum. All spectral features in the theoretical transmission arise from molecular absorption. Since it is free from random variations, the noise in the corrected spectrum is not increased (colour coding as in Fig. 17).

Open with DEXTER

thumbnail Fig. 23

Close up of the prominent water vapour absorption feature between 1.3 and 1.5 μm. Due to the intrinsic noise of the TSS spectrum and the subsequent telluric absorption correction, the noise introduced by the classical method is significantly higher (colour coding as in Fig. 17).

Open with DEXTER

6. Summary and conclusion

We have developed the software package molecfit consisting of routines to fit synthetic atmospheric transmission spectra to science data (see Paper I) and to apply these synthetic spectra as telluric absorption correction to science files. We have extensively tested the software with a large X-Shooter data set to evaluate the performance of the package with two figures of merit, the offset and the small-scale residual parameters, Ioff and Ires, respectively. Moreover, we compared the telluric correction by molecfit with the classical method based on TSS for several science spectra. In the following, we summarise our findings:

  • The telluric absorption correction with molecfit of TSS doesnot introduce systematic offsets in the corrected spectra. Thescatter of Ioff is about 3% of the line strength. The relative small-scale residual strength is about 20% (see Sect. 4.3 and Fig. 5) for the NIR arm, i.e. the quality of the correction is roughly proportional to the strength of the telluric absorption lines. The VIS arm data show results of similar quality.

  • The telluric correction shows a dependency on the number of pixels per FWHM. The small-scale residuals increase with decreasing slit width or FWHM (see Sects. 4.3, 4.4, and Table 6) if the pixel size is kept constant.

  • The quality of the fit crucially relies on the selection of the fitting ranges. In particular, all critical molecular absorption bands existing in the wavelength range of the science spectrum should be covered by corresponding regions (see Sect. 4.5). On the other hand, a few narrow fitting ranges are sufficient to achieve a good quality of the correction over the entire wavelength range. Spreading these narrow fitting regions over the entire wavelength range helps to improve the fit for the line spread function and the wavelength calibration correction.

  • Molecfit offers the possibility to influence the fit by adjustable parameters. The selection of appropriate parameters is crucial for a good result. In particular, the degree of freedom for the fit should the minimised, e.g. the Chebyshev polynomial for the wavelength grid correction has a great influence on the fitting quality. This should be chosen carefully if the fitting ranges are small compared to the entire wavelength range covered by the spectrum as in the case of X-Shooter (see Sect. 4.6).

    thumbnail Fig. 24

    Same as in Fig. 23 but for the prominent water vapour and carbon dioxide absorption feature between 1.8 and 2.0 μm (colour coding as in Fig. 17).

    Open with DEXTER

  • Comparisons with atmospheric profiles measured by a microwave radiometer reveal that the fitting algorithm is very robust with respect to variations of the input atmospheric profile. Thus, the initial profile does not have a large effect as long as it does not deviate extremely from the real one. The PWV value determined with molecfit tends to be too high in the case of very dry observing conditions. However, this does not affect the quality of the telluric corrections.

  • Molecfit is also applicable to low S/N data. However, there may be a loss of quality in the telluric correction for very low S/N observations leading to more residuals. Data with extremely low S/N cannot be fitted reliably. In this case, a TSS can alternatively be used for the fit instead of the science spectrum (see Sect. 4.8).

  • We performed a comparison with the classical method, which is affected by the following problems:

    • (a) The implementation of the TSS continuum determination canlead to significant continuum changes in the corrected spectrum,e.g. by the limited number of fitting points, and subsequentnormalisation of the spectrum.

    • (b) Intrinsic spectral lines of the TSS can change the flux and shape of object lines or mimic additional spectral features in the corrected object spectrum.

    • (c) The intrinsic noise of the TSS observation lead to additional noise in the corrected object spectrum.

    Whenever such a TSS-related problem appears, it can beexpected that molecfit performs significantly better, asdemonstrated in several examples. Moreover, direct fitting of thescience spectrum with molecfit avoids issues related todifferences in the atmospheric conditions for the science and TSSobservations.

  • Molecfit is very robust with respect to single object lines lying in the fitting regions, at least if the regions are not dominated by these features. The telluric correction is only affected marginally in this case. However, care should be taken when molecular bands are expected in the scientific target, which are present in the fitting ranges.

The incorporation of synthetic atmospheric transmission spectra based on theoretical calculations provide a promising way to perform the telluric absorption correction. The highly efficient underlying algorithm of molecfit offers the opportunity to achieve a reasonable and reliable correction without supplementary observations of TSS which are time expensive. In addition, the applicability of molecfit with standard parameters allows already reasonable results. In addition, optimising these standard parameters can be achieved in much less time than optimising the standard method, making molecfit a very efficient tool for the telluric absorption correction. We also successfully applied molecfit to several other ESO instruments covering several wavelength and resolution regimes (see Paper I). Only a slight adaption of instrument dependent parameters is necessary. Any required information can be added in the parameter file if the FITS header of the files does not contain ESO compliant keywords, which are read by molecfit. Although the software is delivered with meteorological data for Cerro Paranal, it provides the capability to create atmospheric profiles appropriate also for

other observing sites. This flexibility makes molecfit a general tool for the telluric absorption correction adaptable to various instruments and observing sites.


10

12 x Intel Xeon X5650@2.67GHz/ 12GB RAM per node.

11

Note that the ozone absorption by the Huggins and Chappuis bands (see Paper I) is usually taken into account by the extinction correction (see e.g. Patat et al. 2011).

12

In the case of the VIS arm, the 0.6′′ slit is replaced with 0.7′′.

Acknowledgments

We thank Sabine Möhler (ESO) for the help with the X-Shooter pipeline and Thomas Krühler (ESO) for providing us the GRB spectrum. This project made use of the ESO archive facility. This study was carried out in the framework of the Austrian ESO In-Kind project funded by BM:wf under contracts BMWF-10.490/0009-II/10/2009 and BMWF-10.490/0008-II/3/2011. This publication is also supported by the Austrian Science Fund (FWF): P26130 and by the project IS538003 (Hochschulraumstrukturmittel) provided by the Austrian Ministry for Research (bmwfw).

References

All Tables

Table 1

Objects used for detailed tests.

Table 2

X-Shooter NIR-arm data set used for the detailed tests (see also Table 1).

Table 3

Molecfit parameter set-up for the telluric absorption correction evaluation of NIR-arm X-Shooter TSS spectra.

Table 4

Wavelength ranges (vacuum) for fitting NIR- and VIS-arm X-Shooter TSS spectra.

Table 5

Sample averages and standard deviations for the transmission-dependent indicators Ioff and Ires of the quality of the telluric absorption correction of NIR-arm X-Shooter spectra.

Table 6

Slit-dependent quality of the telluric absorption correction of NIR-arm X-Shooter spectra.

Table 7

Influence of fitting ranges on the telluric absorption correction of the NIR-arm spectrum displayed in Fig. 2.

Table 8

Influence of changing fitting ranges in a molecular band on the telluric absorption correction of the NIR-arm spectrum displayed in Fig. 2.

Table 9

Influence of wavelength grid correction by a Chebyshev polynomial of order wlc_n on the telluric absorption correction of the spectrum displayed in Fig. 2.

Table 10

Influence of wavelength grid correction by a Chebyshev polynomial of order wlc_n on the telluric absorption correction of the spectrum displayed in Fig. 2 for the fixed best-fit line profile (relres_box = 0.560 and res_gauss = 1.671) as derived from the standard test run.

Table 11

Individual regions used for the telluric absorption correction.

All Figures

thumbnail Fig. 1

Reference X-Shooter NIR-arm model transmission spectrum binned in 1 nm steps for a mean amount of precipitable water vapour (PWV) of 3.1 mm and an airmass of 1. The bins marked by red symbols are classified as continuum. The transmission of these bins ranges from 0.95 to 1.00. The mean value is 0.99. The selected bins are used to derive the systematic offsets (residuals) in the telluric absorption corrected spectra. Bins that belong to four specific transmission ranges corresponding to T = 0.9 (blue), 0.8 (cyan), 0.5 (magenta), and 0.2 (green) are indicated as well. The figure also shows the fitting ranges (yellow bars) that were used by molecfit (see also Table 4).

Open with DEXTER
In the text
thumbnail Fig. 2

Quality of telluric absorption correction for an example TSS spectrum taken with the NIR arm of the X-Shooter spectrograph. The star was observed with a 1.2′′ slit at an airmass of 1.32. The seeing was 0.76′′ and the PWV (as derived from the fitting) was 1.46 mm. Upper panel: telluric absorption corrected spectrum (red) is shown in comparison with the original spectrum (black). Middle and lower panels: mean counts and standard deviation in ADU for a grid of 1 nm bins for the telluric absorption corrected example spectrum. The two subfigures also highlight the pixels for the continuum interpolation (red) and the different transmission ranges for the quality analysis of the telluric absorption correction. For more details, see Fig. 1.

Open with DEXTER
In the text
thumbnail Fig. 3

Indicators Ioff for systematic offsets (upper panel) and Ires for small-scale residuals (lower panel) of the telluric absorption correction for single 1 nm bins of the X-Shooter NIR-arm spectrum shown in Fig. 2. Both quantities are given as a function of transmission. Bins belonging to the different transmission ranges used in the discussion are marked by different colours (see legend and Fig. 1). Small red symbols identify bins which are excluded from further analysis due to either very low transmission or very weak line strength.

Open with DEXTER
In the text
thumbnail Fig. 4

Small-scale residual indicator Ires for 1 nm bins versus the mean model continuum counts in ADU for the five fitting ranges. The red dashed line shows the selection criterion for sufficiently high S/N for the detailed analysis.

Open with DEXTER
In the text
thumbnail Fig. 5

Indicators Ioff for systematic offsets and Ires for small-scale residuals of the telluric absorption correction for 1 nm bins. Only data points of the filtered subsample are shown.

Open with DEXTER
In the text
thumbnail Fig. 6

Fraction of NIR-arm bins with standard deviations of the residuals of the telluric absorption correction greater than or equal to 5% of the object continuum as function of the best-fit PWV in mm. The red regression line has a slope of 0.15 per dex.

Open with DEXTER
In the text
thumbnail Fig. 7

Best-fit PWV obtained by molecfit in mm as function of the observing date in years.

Open with DEXTER
In the text
thumbnail Fig. 8

Small-scale residual indicator Ires for 1 nm bins versus the FWHM of the line profile in pixels. As indicated by the legend, the different colours correspond to different slit widths.

Open with DEXTER
In the text
thumbnail Fig. 9

Mean counts in ADU and standard deviation relative to mean counts for a grid of 1 nm bins of the telluric absorption corrected example spectrum shown in Fig. 2 for three different sets of fitting ranges. The black spectrum equals the one in Fig. 2 and corresponds to the standard set-up of windows described in Table 4. The red and the green spectra were calculated by only using a single fitting range (see legend and Table 7).

Open with DEXTER
In the text
thumbnail Fig. 10

Mean counts in ADU and standard deviation relative to mean counts for a grid of 1 nm bins of the telluric absorption corrected example spectrum shown in Fig. 2 for different orders of the Chebyshev polynomial for the wavelength grid correction and fixed line profile (see legend and Table 10).

Open with DEXTER
In the text
thumbnail Fig. 11

PWV in mm versus the best-fit scaling factor for the input water vapour profile (output value for relcol parameter; see Table 3).

Open with DEXTER
In the text
thumbnail Fig. 12

Relative difference of the telluric absorption corrected spectra of the TSS Hip039634 (IWV = 1.5 mm), one corrected with a LHATPRO, and one with a GDAS based profile (in per cent). ctsH = counts of the LHATPRO based telluric absorption corrected spectrum; ctsG = GDAS counterpart.

Open with DEXTER
In the text
thumbnail Fig. 13

Comparison of the relative water vapour column scaling factors relcol derived with the standard GDAS/MeteoMonitor model and the LHATPRO measurements for 549 TSS spectra.

Open with DEXTER
In the text
thumbnail Fig. 14

X-Shooter NIR-arm spectrum of γ-ray burst GRB 130606A at z = 5.913. The graphs show the wavelength range from 1.15 to 1.16 μm, which is affected by both telluric and some intrinsic absorption lines arising at a redshift z = 3.4515 (Hartoog et al. 2014). Upper panel: uncorrected spectrum. Middle panel: transmission spectrum as calculated by molecfit. Lower panel: corrected spectrum.

Open with DEXTER
In the text
thumbnail Fig. 15

X-Shooter NIR-arm spectrum of the Seyfert 2 galaxy PKS1934-63 (z ~ 0.183). Upper panels of a), b), and c): uncorrected and the molecfit corrected spectrum in different wavelength ranges. This observation has a low mean ADU level of 462 counts. The displayed transmission curve (lower panels) is based on a fit in the standard fitting regions (yellow areas).

Open with DEXTER
In the text
thumbnail Fig. 16

Upper panel: telluric standard star spectrum Hip084409 (test data set #3, see Tables 1 and 2) and the fitting method for extracting a transmission curve: a cubic spline fit to base points (blue dots in upper panel) is used to derive the continuum of the telluric standard star. This fit is then used to normalise the spectrum to achieve the corresponding transmission (lower panel). We divided the spectrum into several wavelength regions based on the amount of absorption. Regions #1 through #8 are heavily affected by absorption, in contrast to regions (a) through (h) (see Table 11). Each region was corrected individually with molecfit and IRAF. Middle panel: resulting transmission curve including some molecular absorption features. Lower panel: theoretical transmission curve achieved with molecfit.

Open with DEXTER
In the text
thumbnail Fig. 17

Comparison of the telluric absorption correction methods: uncorrected object spectrum (black), molecfit corrected spectrum (red line), and the spectrum corrected with the classical method (green) in a region with minor atmospheric absorption (CO2 band).

Open with DEXTER
In the text
thumbnail Fig. 18

Upper panel: example for poor continuum reproduction by the classical method (colour coding as in Fig. 17). Although the prominent CO2 absorption feature is well corrected by both methods, significant variations in the continuum are visible in the spectrum corrected by the classical method. Intrinsic object lines redwards of 2 μm are well reconstructed by both methods. Lower panel: relative residuals of the corrected spectra when object lines are excluded from the fit in region #5 (blue areas in upper panel).

Open with DEXTER
In the text
thumbnail Fig. 19

Panel a): Brackett series from V921-Sco, a B(e) star. Panel b): zoomed in region with bad continuum correction by the classical method (colour coding as in Fig. 17).

Open with DEXTER
In the text
thumbnail Fig. 20

Paβ line visible in the spectrum of the PN IC1266. Upper panel: although the line is located in a wavelength range with minor atmospheric absorption, it is not well corrected with the classical method. An intrinsic wavelength shift is probably introduced by the radial velocity of the object (colour coding as in Fig. 17). Lower panel: transmission derived with the classical method (green line) shows a broad dip leading to a significant higher flux and different line shape in the corrected spectrum. The transmission derived with molecfit (red) does not show this feature.

Open with DEXTER
In the text
thumbnail Fig. 21

Example of an overcorrection by spectral features arising from the Brackett lines (H(410), H(411), and H(412)) visible in the TSS. In the molecfit based correction such features do not occur (colour coding as in Fig. 17).

Open with DEXTER
In the text
thumbnail Fig. 22

Upper panel: detailed zoomed in region with minor atmospheric absorption. Due to variations caused by the spline fit and noise in the TSS spectrum, the classical method introduces continuum variations and noise to the corrected spectrum. All spectral features in the theoretical transmission arise from molecular absorption. Since it is free from random variations, the noise in the corrected spectrum is not increased (colour coding as in Fig. 17).

Open with DEXTER
In the text
thumbnail Fig. 23

Close up of the prominent water vapour absorption feature between 1.3 and 1.5 μm. Due to the intrinsic noise of the TSS spectrum and the subsequent telluric absorption correction, the noise introduced by the classical method is significantly higher (colour coding as in Fig. 17).

Open with DEXTER
In the text
thumbnail Fig. 24

Same as in Fig. 23 but for the prominent water vapour and carbon dioxide absorption feature between 1.8 and 2.0 μm (colour coding as in Fig. 17).

Open with DEXTER
In the text

Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.

Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.

Initial download of the metrics may take a while.