Issue |
A&A
Volume 515, June 2010
|
|
---|---|---|
Article Number | A35 | |
Number of page(s) | 18 | |
Section | Astronomical instrumentation | |
DOI | https://doi.org/10.1051/0004-6361/201014022 | |
Published online | 04 June 2010 |
P3D: a general data-reduction tool for fiber-fed integral-field spectrographs
,![[*]](/icons/foot_motif.png)
C. Sandin1 - T. Becker1 - M. M. Roth1 - J. Gerssen1 - A. Monreal-Ibero2 - P. Böhm1 - P. Weilbacher1
1 - Astrophysikalisches Institut Potsdam (AIP), An der Sternwarte 16, 14482 Potsdam, Germany
2
- European Organisation for Astronomical Research in the Southern
Hemisphere (ESO), Karl-Schwarzschild-Straße 2, 85748 Garching bei
München, Germany
Received 8 January 2010 / Accepted 11 February 2010
Abstract
The reduction of integral-field spectrograph (IFS) data is
demanding work. Many repetitive operations are required to convert raw
data into, typically, a large number of spectra. This effort can
be markedly simplified through the use of a tool or pipeline, which is
designed to complete many of the repetitive operations without human
interaction. Here we present our semi-automatic data-reduction tool P3D, which is designed to be used with fiber-fed IFSs. Important components of P3D
include a novel algorithm for automatic finding and tracing of spectra
on the detector and two methods of optimal spectrum extraction in
addition to standard aperture extraction. P3D also provides tools to combine several images, perform wavelength calibration and flat field data. P3D is
at the moment configured for four IFSs. To evaluate its
performance, we tested the different components of the tool. For these
tests we used both simulated and observational data. We demonstrate
that a correction for so-called cross-talk due to overlapping spectra
on the detector is required for three of the IFSs. Without such a
correction, spectra will be inaccurate, in particular if there is
a significant intensity gradient across the object. Our tests showed
that P3D is able to produce accurate results. P3D is
a highly general and freely available tool. It is easily extended
to include improved algorithms, new visualization tools, and support
for additional instruments. The program code can be downloaded from the
P3D-project web site http://p3d.sourceforge.net.
Key words: methods: data analysis - methods: observational - techniques: spectroscopic
1 Introduction
With integral integral field spectrographs (IFSs), an extended area on the sky can be spectroscopically mapped, under the same observing conditions, in one single exposure. To fit all simultaneously observed spectra onto the detector, the field-of-view of the integral field unit (IFU) that provides the spatial sampling on the sky is relatively small. The footprint of IFUs typically ranges from several arcsec, e.g. PMAS, to about one arcmin for VIMOS. In an alternative configuration, IFUs are used to map a much larger area, but with sparse sampling, e.g. PPAK and VIRUS-P. In Table 1 we list several of the existing fiber-fed IFUs, the telescope they are mounted on, and the names of the corresponding pipelines. See Bershady (2009) for a complete list. Regardless of the actual IFS configuration, the raw data of fiber-fed IFSs always consist of hundreds, or even thousands, of spectra per exposure. The reduction of these data consists of processing each spectrum individually and is therefore highly repetitive work. A general data-reduction tool is highly desirable, one which automates the reduction steps yet allows the user to interactively inspect and optimize parameters when required. The purpose of P3D is to provide such capabilities, thereby facilitating the scientific exploitation of IFSs.
Table 1: A list of fiber-fed IFUs and their respective data-reduction pipelines.
Special purpose data-reduction pipelines exist for most IFUs. Two
additional reduction packages that are more general in their
functionality than the ones listed in Table 1 and are suitable to use with fiber-fed IFUs, are R3D (Sánchez 2006, hereafter S06) and IRAF.
These tools, however, require significant amounts of time-consuming
manual interaction to get the best out of them. While developed
initially for the PMAS spectrograph, the data-reduction tasks of P3D (and P3D_ONLINE) are very generally formulated and are equally applicable to data obtained with other fiber-fed IFSs. The suitability of P3D
for several other IFSs is demonstrated with a variety of scientific
results at an early stage, when no such tools were available yet for
those instruments shortly after commissioning, e.g. MPFS (Roth et al. 2004; Lehmann et al. 2005; Fabrika et al. 2005) or VIMOS (Villar-Martín et al. 2006; Monreal-Ibero et al. 2005).
In this paper we present a generalized version of P3D, which is a complete rewrite of the previous version by Becker (2002). This general IFS reduction tool includes support for PMAS/LARR, PMAS/PPAK, VIRUS-P, and SPIRAL and can be readily extended to additional IFUs. Key features of P3D are
- a single, freely available, and easy to install, program package with support for several IFSs and computing platforms;
- calculation and propagation of errors through all steps;
- the option to choose between aperture extraction and two methods of optimal extraction;
- a possibility to store information about all performed operations in a log file;
- interactive and integrated visualization tools to allow the user to examine intermediate and final products.
This paper is laid out as follows. In Sect. 2 we first describe the goals and setup of P3D. The data-reduction algorithms and their implementation are thereafter explained in Sect. 3. We discuss and analyze the outcome of the tool, and compare it with the corresponding outcome of IRAF, for some of the tasks in Sect. 4. Finally, we close the paper with our conclusions in Sect. 5.
2 About the objective and setup of P3D
Our goal with P3D is to provide a general, flexible, fast, and reliable tool for reduction of fiber-fed IFU data. P3D is not intended to be a tool that handles every possible task from data reduction to data analysis; instead, it focuses on the tedious and repetitive tasks that are required to convert raw data into wavelength-calibrated spectra, cf. Sect. 3. We also want to provide a user-friendly tool where the required input from the user is kept to a minimum. It is recommended, however, that the user have a basic knowledge of the data-reduction process and of the related numerical problems and methods, to allow a full exploitation of all the benefits of P3D.
The collection of routines that make up P3D were from the start (Becker 2002; Roth et al. 2005) written in the Interactive Data Language
(IDL). Supplementary routines, which are mainly related to file-IO, are
used from the publicly available toolkit astro-lib of NASA
. The MPFIT routine of Markwardt (2009)
is also used when fitting line profiles (for the optimal spectrum extraction). The current version of P3D -
a public release under GPL-v3 - is a complete
rewrite of earlier versions and remains a graphical tool (GUI),
although reduction tasks can also be completed without the GUI. We
redesigned the tool to work with all platforms that are supported by IDL
(version 6.2 or higher). We stress that P3D
can be used with full functionality without an IDL license, using
the IDL Virtual Machine (VM). Currently, all required scripts are
provided to use the IDL-VM with all UNIX-type platforms. The
installation procedure consists of setting up IDL to include the files
of P3D and the supplementary routines in the path of IDL. The software, updates, documentation, and tutorials can be found at the P3D-project web site, http://p3d.sourceforge.net.
All routines and their input and output parameters are documented. When P3D
is run with the GUI, the functionality of each interactive part is
described with a comment in a status indication field, and information
on all activity is optionally saved to a log file. We minimized the
run-time by replacing time-consuming FOR-loops with intrinsic
IDL functions. All parts of the GUI can be used with screen sizes
from 1024
600 and larger. Moreover, the program currently consists of about
45 000 lines of code. Any redundancy is kept to a
minimum by adhering to the ``Don't Repeat Yourself'' principle of
coding (Hunt & Thomas 1999).
Our main priority is to correct algorithm questions, which lead to
erroneous scientific output, as soon as possible. Avoidable issues,
which are usually related to different parts of the GUI, are corrected
as time permits.
Currently P3D is configured for four IFUs: the lens array (LARR) and PPAK IFUs of the PMAS instrument (for both the old
CCD and the new
CCD, Roth et al. 2005; Kelz et al. 2006)
that is mounted on the 3.5 m telescope at Calar Alto, the
VIRUS-P IFU (for both bundle 1 and the newer bundle 2, Hill et al. 2008) that is mounted on the 2.7 m Harlan J. Smith telescope at the McDonald Observatory, and the SPIRAL IFU (see e.g. Smith et al. 2004)
that is mounted on the 3.9 m Anglo-Australian telescope. Support
for additional instruments can be added through appropriately formatted
configuration files.
3 Components of the data reduction
P3D contains a set of routines to do the following five tasks: create a master bias, trace all spectra on the detector, create a dispersion mask, create a flat field, and extract spectra in object data. We now describe these tasks in more detail.
For each task several raw data images can be combined by P3D
in order to increase the signal-to-noise and remove cosmic rays.
The default combination method is to use a stack of at least three
images, where the minimum and maximum values of each pixel are thrown
away before calculating a (min/max-) average of all images.
Optionally, an average or a median can be used. Prescan- and
overscan-regions in the raw data are only removed immediately before
spectra are extracted, and this must be accounted for if external tools
are used to create intermediate products. Spectrum extraction can be
done using either aperture extraction, our own so-called modified
optimal extraction, or a multi-profile deconvolution optimal
extraction (cf. Sect. 3.2). Both optimal extraction methods are able to correct for so-called cross-talk due to overlapping spectra. Currently, P3D does not include tasks to apply a bad pixel map, remove scattered light, subtract the sky, perform flux calibration, compose dithered or mosaiced combined frames, or any kind of binning of spectra.
In the first step a master bias image is created by combining a set of at least three bias images. This master bias is subtracted from the raw data in all consecutive steps. In the second step the position of each spectrum is determined along the dispersion axis with a well-illuminated calibration exposure, using a continuum lamp or twilight flats. The resulting trace mask is used in all consecutive steps when spectra are extracted. Finally, cross-dispersion profiles are calculated for all spectra and wavelengths in order to allow optimal extraction.
In the third, optional step a dispersion correction is determined for each spectrum, using one or several arc lamp exposures. An extracted flat field image is created in the fourth, also optional step to correct for wavelength-dependent variations of every spectrum and for differences in the fiber-to-fiber throughput. In the fifth, and final step all spectra are extracted from object exposures, optionally applying first the dispersion correction and thereafter the flat-field correction.
To illustrate typical properties of IFU raw data, we show a section of object data in Fig. 1, together with the corresponding continuum lamp data, which were taken with the PPAK IFU. Each horizontal line marks the position of a spectrum, the two brightest lines (that are only seen in the lefthand side image) belong to two out of fifteen calibration fiber spectra of PPAK (these are not used in P3D). The strong variation in emission line intensities of different spectra (on the cross-dispersion axis) stems from the spatially irregular mapping of fibers in PPAK (cf. Kelz et al. 2006). The two data sets that are shown next to each other illustrate their interdependence: spectrum positions in the object data should match those in the continuum (calibration) data well in order to extract spectra properly.
![]() |
Figure 1: This figure shows two
adjacent PPAK raw data sections highlighting one group of
32 spectra. We show continuum lamp data on the lefthand side and
object data, about H |
Open with DEXTER |
Table 2: General and instrument-specific parameters of the data-reduction algorithms of P3D.
In the following we assign a symbol to all variable parameters so
that they can be found easily inside the program code. For a quick
reference we collected all symbols and their respective default value,
in Table 2,
the meaning of the parameters are introduced step-by-step below. The
CCD readout binning parameters are, furthermore, denoted
by bin
(dispersion axis) and bin
(cross-dispersion axis), and these parameters are set to 1 or 2.
Next we describe how P3D handles the separate data-reduction tasks. Our novel spectrum tracing algorithm is described in Sect. 3.1. Thereafter we describe our approach to spectrum extraction in Sect. 3.2, how we wavelength calibrate the data in Sect. 3.3, how we flat field data in Sect. 3.4, and how object spectra are extracted in Sect. 3.5.
3.1 An automated method for finding and tracing spectra in IFU raw data
Before spectra can be extracted, they must be found and their extent be traced along the dispersion axis on the detector. To allow accurate tracing, a continuum lamp exposure should be used where spectra are clearly visible across the full wavelength range. Most IFU-instruments are, moreover, affected by flexure, which results in changing positions of spectra on the detector as the telescope moves over time, and examples of instruments without such flexure are VIRUS-P and INTEGRAL. If the flexure is significant, it is important to trace with a continuum lamp exposure, which is sampled close in time to the object exposure, in order to extract data from the correct region on the detector. It may also be an option to calculate one trace mask and then shift it using knowledge about the shifted positions of selected lines of arc images.
We present an automatic algorithm here for spectrum tracing that involves several steps to assure that both all expected spectra of an instrument are found and traced accurately and that no noise feature is kept as a spectrum. The method is general and should require few, if no modifications once it is set up for an instrument. No assumptions are made regarding exact positions of spectra on the detector. Instead information is required about their expected number, their orientation, and their ordering. Cosmic ray hits could, if numerous, have some effect on the outcome, in which case they should be removed in advance.
The algorithm is split into two steps. At first all n spectra are searched along the cross-dispersion (spatial) axis for one pixel, or a set of averaged pixels, on the dispersion axis. Thereafter they are traced along the dispersion axis for the remaining pixels. All instrument-specific default parameter values of this procedure are shown in Table 2 for all considered IFUs. We next describe the two tracing steps separately.
3.1.1 Step 1: finding the spectrum positions
At first, the local maxima, representing positions of individual
spectra, are located on the cross-dispersion axis; in this
process, a set of adjacent pixels is used on the dispersion axis,
which is typically centered on the middle pixel of the CCD. It is
assumed that spectra are fairly well aligned with either axis on the
detector. The following condition must hold for any local maximum,

where f(y) is the intensity distribution across the cross-dispersion axis (y) for every pixel j, and





Next, in step 1b, knowledge of the spectrum pattern separation is
used to filter out spectra from the sequence of local maxima of
step 1a. The main assumption is that consecutive spectra are
separated by an instrument-specific distance (pitch) d(y).
The first maximum in the sequence is used as a starting point of a
sequence. To match a sequence, the distance to the subsequent
maximum must fulfill

where m








![]() |
Figure 2:
In this figure we illustrate how adjacent spectra are separated for four IFUs: a) PMAS-LARR (using bin |
Open with DEXTER |
In step 1c, a cross-correlation is made between the sequence of maxima
of step 1b and a pre-defined instrumental pattern that specifies
the expected separation and number of spectra. This pattern is defined
as a list of spectrum gaps, assuming spectra are separated as described
in step 1b (setting m = 1). For example, if two groups of spectra are separated by a distance corresponding to 2 d,
then this corresponds to one entry in the list of gaps. After the
cross-correlation, positions of missing spectra due to e.g. dead
or unused fibers are inserted separately. Those positions, which could
not be identified in steps 1a and 1b, are interpolated or
extrapolated from the position of the nearest found spectrum.
In this way the number of spectrum positions in the returned
sequence is always as expected, viz. n. This is a
useful property when working with data of IFUs such as VIMOS that has
many fibers with poor throughput. The probability that a noise feature
is identified as a spectrum is negligible after this third step.
Finally, more accurate positions of the identified sequence of
spectra are calculated by weighting with the cross-dispersion profile
of the data. In this weighting the width of the used spectrum
section is
pixel. The full-width at half-maximum (FWHM) of the Gaussian profile that is used in the weighting is
pixel. This procedure is iterated
times in every case.
In Fig. 3 we show an example of the outcome of step 1 of the tracing algorithm for calibration data, which were sampled with the PPAK IFU. For a direct comparison we show the same section of the spectrum that we present in Fig. 1. Note that 35 out of 36 visible spectra were identified in steps 1a and 1b. The position of spectrum 225 is hidden in the much stronger spectrum 224. The position of this spectrum was determined in step 1c, where the spectrum pattern defined its location (as d pixels distant from spectrum 224). All spectra but those of the calibration fibers 192 and 224 are to a lesser degree affected by cross-talk, cf. Sect. 4.2.
![]() |
Figure 3:
The result of step 1 of the tracing algorithm for continuum lamp data of the PPAK IFU, cf. Sect. 3.1.1. In this case the used CCD binning setup was
|
Open with DEXTER |
3.1.2 Step 2: tracing along the dispersion axis
In a second step we calculate spectrum positions for all wavelength bins, starting with the resulting weighted positions of step 1. If spectra are traced individually along the dispersion axis, there is a risk that some spectra could be lost, and because of effects of pixel subsampling, an oscillating pattern of individual spectra might be found in the trace mask. The amplitude of the oscillations decreases with the profile width and becomes significant with smaller widths. The associated oscillation period depends on the angle between the spectrum and the row of pixels and corresponds to the number of pixels in a row of pixels hosting a maximum. If positions are smoothed, using appropriately selected smoothing parameters, these oscillations can be made smaller, cf. Sect. 4.1.
We used the following approach. At first the data set is resized, by a defined factor f, on the dispersion axis, in order to reduce noise effects. In this process the value at each position is averaged over pixel.
For every bin on the dispersion axis, we then calculate weighted
spectrum positions, starting with the already known position of a
neighbor wavelength bin. To remove subpixel-sampling effects the
positions are thereafter smoothed twice using a boxcar of width
pixel,
first using a median and then using an average. After all positions are
calculated we smooth the positions on the dispersion axis, using a
boxcar of width
pixel.
Finally we calculate spectrum positions for all pixels on the dispersion axis. Using the smoothed and rebinned data set the remaining positions are calculated by linear interpolation. Values at either end are extrapolated.
3.1.3 Adjusting the automatic tracing procedure
The philosophy of this tracing algorithm is that it be automatic, and no modifications of the default values of Table 2
should be required. However, it may turn out in some circumstances
that the algorithm is unable to correctly identify all spectra.
In this case, the recommended procedure is to first vary
to see if the number of spectrum misfits can be made smaller (step 1a). A second option is to vary
,
and maybe also
,
in order to find outliers (step 1b). If the number of
misfits is large, it may be necessary to add or remove one entry
in the list of spectrum gaps (step 1c). If the spectrum
separation parameter d is changed, it is in any case necessary to modify this list.
3.2 Introducing the three spectrum extraction methods
The flux in every wavelength bin of any spectrum on the detector is distributed in a profile (aperture) on the cross-dispersion axis. The shape and extent of the profile depends on the instrumental setup. If many spectra are squeezed onto the surface of the detector, as is often the case with IFSs, there is likely to be some overlap between profiles of adjacent spectra, resulting in cross-talk. The smaller the spectrum separation (pitch) and the larger the spectrum width, the greater the overlap. By selecting an accurate method of spectrum extraction, it is mostly possible to separate overlapping spectra well and attain both accurate and precise values of the flux. Such an approach can, however, be very time-consuming, which is why it is worth testing whether a simpler method suffices.
From here on, we assume that the data set ,
which contains spectra, is bias-subtracted. The variance in the flux for every pixel k in
is then (cf. e.g. Howell 2006),
where








Next we describe the standard aperture extraction in Sect. 3.2.1. Thereafter we introduce our modified optimal extraction (MOX) method in Sect. 3.2.2, and how we implement the multi-profile deconvolution (MPD) method in Sect. 3.2.3.
3.2.1 Standard aperture extraction
In our first approach we use an aperture of pre-defined width with a
top-hat (box-car) profile that is set to 1 within the aperture,
and 0 outside. This method is also referred to as tramline
extraction (see e.g. Sharp & Birchall 2010,
hereafter S10). We account for aperture boundaries inside the pixels by
adding fractions of the flux of such pixels. The integrated flux f and the corresponding variance
of every spectrum i and the contributing pixels j(i) of every profile, for the wavelength bin
,
are
We present the instrument-specific default aperture widths,

Although almost all flux can be collected with an aperture extraction for IFUs such as LARR, this is still an inefficient method if fluxes are low. If this is the case the contribution of readout-noise from the outer pixels of an aperture can become significant, and even dominate flux errors, cf. Sect. 4.4. Since this is the fastest method of spectrum extraction, it is, nevertheless, the default method of P3D.
3.2.2 Modified optimal extraction (MOX)
Optimal extraction (Horne 1986, hereafter H86) is a more accurate method than aperture extraction. In this method line profiles are used to weight flux from the separate pixels across apertures. With such line profiles it is also easier to filter out pixels, which are hit by cosmic rays. Although the process of extraction is straightforward, the calculation of line profiles is demanding with IFUs. The most direct approach requires a separate profile to be calculated for each spectrum and wavelength bin. This is not only a time-consuming process, but also makes it difficult to correct for noise effects and cross-talk. S06 attempts such an approach and draws a similar conclusion. The number of free parameters may be almost equal to the number of pixels, and possible effects of pixel subsampling (see above) are neglected.
Starting at one wavelength we simultaneously fit a group of line profiles, using a pre-selected function. Doing this we use the same continuum image as was used to calculate traces. The number of spectra ml in each group l depends on the instrument setup. We show the ranges of ml which we use with the four IFUs in Table 2. The function can be selected to be a Gaussian function, a Lorentzian function, an approximative Voigt profile, or a double Gaussian function. As we currently ignore any broad component due to scattered light, it appears adequate to use a single Gaussian profile for all IFUs.
Since the profile width typically changes slightly across the
detector, it seems inappropriate to fit all spectra
simultaneously, and such an approach is also computationally much more
time-consuming. For each group of spectra we (by default) assume
fixed profile center positions using the trace mask. The center
positions can be fitted as well, if necessary. With a
Gaussian line profile the result of the fit consists of the spectrum
width
,
l intensities, and the zero-level and the gradient of the
linear fit to the background. Thereafter the same procedure is carried
out for a set of additional wavelength bins, and as a last step we
interpolate the profile parameters for all intermediate wavelength bins
using cubic splines. The number of wavelength bins used to calculate
the fits,
,
should be selected to allow a reasonable interpolation for intermediate
wavelengths using such splines. Finally, before spectrum extraction,
the profile parameters are used to calculate pixel-based line
profiles for every spectrum and wavelength bin. Each profile is
thereafter normalized.
In comparison to ``plain'' optimal extraction, our modified approach
allows a correction for overlapping spectra (i.e. cross-talk).
Including masking of pixels, which are hit by cosmic rays, and
correction for cross-talk, the equation of the modified optimally
extracted flux becomes
where P is the normalized line profile, M a profile mask (M










After



![]() |
(5) |
and iterate the solution n2 times. Again, the default is n2 = 1 following H86.
Next we describe how we remove cosmic rays and correct the integrated flux for cross-talk. Both methods require an inspection of the resulting outcome. These are therefore options that must be switched on separately in P3D.
Removal of cosmic ray hits.
To remove cosmic ray hits we follow the approach of H86 and first iterate the integrated flux (Eq. (3)) at most

by setting






Correcting extracted flux when there is overlap between nearby spectra on the CCD.
To correct for cross-talk, we iterate the spectrum extraction. In every iteration we first calculate a total profile across all ml spectra of group l. Thereafter we calculate a fractional profile






3.2.3 Multi-profile deconvolution optimal extraction (MPD)
In addition to the modified optimal extraction, P3D
can also use the multi-profile deconvolution method of S10.
In comparison to our method above, all line intensities at one
wavelength are here solved for simultaneously. Specifically, for every
group of spectra the method is to minimize the residual
![]() |
(6) |
Assuming

for the intensities


Considering the variance we calculate it using a similar approach by minimizing the residual
![]() |
(8) |
Assuming

for the variances


The other two spectrum extraction methods use a relatively limited size of the aperture. In this method, we use the aperture width

where


Following the approach of S10, the systems of equations (Eqs. (7) and (9)) are solved for
using a tri-diagonal solver if
= 1. For
1
there is a choice of solving the equations using either a
sparse-diagonal matrix solver or a singular-value decomposition solver.
3.3 Preparing a dispersion solution
Depending on the optical path through instrument and fibers, the resulting spectra of separate spatial elements of an IFU are shifted and stretched relative to each other. An image with extracted spectra of an arc lamp exposure shows emission lines as curves across the cross-dispersion axis. To find a dispersion solution, which is unique to an IFS exposure, all spectra should be aligned and stretched to use the same starting wavelength and size of wavelength bins.
![]() |
Figure 4: This figure shows three extracted arc frames of: a) LARR, b) LARR after the dispersion solution was applied, and c) VIRUS-P. Colors are modified to enhance various features. In each panel the vertical line with a disk indicates a preset reference spectrum and wavelength bin. In panel a) the arrow points at the merging point of a shorter (1s; left part) and a longer (10s; right part) exposure. This straight line is curved in panel b) (left arrow). In panel b) the right arrow points at an emission line that was outside the calibration range. In panel c) the left arrow indicates the curved shape of a line in the line mask, after the line geometry has been calculated. The two right arrows indicate a mismatch between the line mask and the data for redder wavelengths. For more details see Sect. 3.3. |
Open with DEXTER |
To calculate a dispersion mask, P3D requires a list of lines with known wavelengths, some information about the expected wavelength range of the used setup, and an arc lamp image. In brief, the list of lines is first modified to match pixel positions of corresponding entries in the data. Thereafter a polynomial is fitted between pixel positions and corresponding wavelengths. Next we describe the individual steps in more detail.
In a first step, a list of lines with well known wavelengths is
selected, either automatically using information in the arc image file
header or as defined by the user. For PMAS, SPIRAL, and other
instruments where the canonical reflective grating equation applies,
an initial estimate of the wavelength range is calculated using
information about the instrumentation setup in the data header. For
VIRUS-P the wavelength range is pre-defined as 3620-5910 Å.
In a second step a line mask curvature is determined by calculating
positions for one or two lines in the arc frame. The algorithm to do
this involves locating the maximum position of one line in one
spectrum, and then tracking this maximum through all other spectra.
Emission lines of separate fibers in PMAS data using the
-CCD
are, to first order, only shifted in wavelength relative to each
other. In this case we found that it suffices to select one
emission line in the data to calculate the curvature accurately enough.
VIRUS-P, SPIRAL, and PMAS using the
-CCD
are different, for these instruments the dispersion varies across
the IFU surface. In this case we select two clearly separated
emission lines in the data, in order to calculate both the
curvature and the change of dispersion.
We present examples of curved arc lines in the blue wavelength range for LARR in Fig. 4a (for lines of mercury) and for VIRUS-P (bundle 2) in Fig. 4c (for lines of cadmium and mercury). To illustrate the change in dispersion of VIRUS-P we calculated the curvature using one line in this case, instead of two. As Fig. 4c shows, the curvatures of lines in the line list badly matches the data in the redder part of the image. If two lines are used instead to calculate the curvature, the match is excellent across all spectra.
In a third step the line mask is shifted along the dispersion
axis in order to achieve a rough match between entries of the line mask
and lines in the arc image. If lines cannot be matched across the
entire dispersion axis, the constant pre-estimated dispersion can be
changed manually. With the PMAS
-CCD,
in particular, the dispersion changes between the blue and the red
ends of the detector. In this case it is necessary to fit a
preliminary non-linear dispersion solution to an interactively selected
set of lines. If the selected line mask also contains saturated
lines or if there are more entries in the input line mask than are
visible in the data, such entries can be removed by hand in a fourth
step.
More precise pixel positions of entries in the line mask are
then calculated by correlating every line in the line mask with the
data. In this fifth step more precise pixel positions are by default
calculated with a Gaussian function fitting. Alternatively they
can be determined using a much faster iterated average-weighting
scheme. The region that is searched for an intensity maximum normally
spans a range of pixel
on the dispersion axis. If emission lines are too tightly packed,
in either the line mask or in the data, this step may fail;
in this case, it is recommended to remove a few entries in
the line mask.
The sixth, and final, step is where a dispersion mask is created. A (linear) polynomial of pre-defined order p is fitted to the pixel positions at all wavelengths, for every spectrum, in the line mask. The polynomial order can be set to any value, but the default is to use a low order with p = 4. Residuals of the fits are stored to the data reduction log file and can also be inspected in a plot. The dispersion mask is saved as p+1 fitting parameters of every spectrum.
Finally, we show an example of how an extracted arc lamp frame appears before and after the dispersion solution is applied in Figs. 4a and b. The extremely noisy line in the right part of Fig. 4b lies outside the range of selected lines, which were used in the creation of the dispersion mask, and was therefore inadequately calibrated.
3.4 Flat fielding the extracted IFU data
The optical path and transmission efficiency of individual spatial elements across an IFU typically vary. Differences appear both as wavelength-dependent variations and as a variance in the fiber-to-fiber throughput. For multiple-detector instruments, such as VIMOS, there is also a likely difference in the detector-to-detector throughput. After spectra are extracted and wavelength-calibrated P3D can correct for these variations by normalizing the data with an extracted flat-field image, which combines the required corrections.
A correction for a variable fiber-to-fiber throughput is
applied by dividing each spectrum of a flat field image with its mean
spectrum. After we correct for wavelength-dependent variations,
each spectrum is at first smoothed across the dispersion axis
using a box-car of width
pixel. Thereafter it is replaced with a
order
(linear) polynomial fit. Finally, the flat field is normalized
with the mean value of all elements. Only non-zero elements are used
with these operations. This smoothing minimizes the amount of noise
that is added to the flat-fielded spectra (see e.g. Becker 2002; Roth et al. 2005). It can be switched off, if required (by setting
= 0 and
= 0).
Before a normalized flat field is created P3D, by default, first calculates a trace mask for this task using the same image. If twilight exposures of flexure-affected instruments are used as flat fields, this is an important aspect as it assures that the proper traces are used. The extracted flat field can, if required, be wavelength-calibrated using a separate dispersion mask.
3.5 Object extraction
The only prerequisite for extracting spectra of object data is a trace mask. Cosmic ray hits can currently be removed either by allowing P3D to combine a set of raw data images or by using the cosmic ray removal option of the modified optimal extraction procedure for separate images. Alternatively, for separate images they could be removed in advance, outside P3D, with, for example, the approach of van Dokkum (2001, who calls his routine L.A. Cosmic) or Pych (2004). In principle, a cosmic-ray removal option can also be added to the multi-profile deconvolution extraction procedure. We do not recommend this approach, however, since it is difficult to remove such hits efficiently while keeping all the flux. Instead, for a future version of P3D we propose using a mask that indicates cosmic-ray affected pixels (calculated using e.g. L.A. Cosmic). The intensity error of the masked pixels are set to a high value before the extraction, and will thus be given a minimal weight using either of the optimal extraction routines. Errors are, furthermore, calculated and stored in a separate file for all intermediate products, but only if a master bias is specified.
After their extraction, spectra are wavelength calibrated if a dispersion mask is specified. If the object data contains sky emission lines, P3D optionally calculates a shift of the dispersion mask based on the known wavelengths of those lines. The offset is calculated by first fitting all present sky emission lines in all spectra with a Gaussian profile. The offset is then taken as the median of the difference between the center positions and the expected wavelengths. The error of wavelength-calibrated spectra should, moreover, be considered a lower limit since pixel values are not cross-correlated when interpolating the dispersion solution to a common base wavelength.
P3D stores all data in row-stacked-spectra (RSS) format, although final spectra can optionally also be saved in the E3D-format (see e.g. Kissler-Patig et al. 2004). When the RSS-formatted file is used to view or analyze the data further, outside P3D, it is necessary to use a separate table specifying the positions of the spatial elements. The tables of the P3D-distribution in this case provide all needed information. If the data need to be corrected for effects of differential atmospheric refraction, we recommend making this correction before the data is flux-calibrated. The approach of Filippenko (1982) provides the most straightforward approach, as it only requires information about observing conditions. The required information is mostly found in the data header. For more details on this issue see Sandin et al. (2008) and the spatial interpolation procedure of Arribas et al. (1999).
4 Program validation
In this section we present the outcome of our tests of P3D. Primarily we used simulated data since the outcome is then known. In our study of properties of the trace mask and the dispersion mask we also compared outcome of P3D with corresponding outcome of IRAF for data of the LARR IFU. A description of the observational setup for used data, in this case, can be found in Relaño et al. (2010). The IRAF data-reduction follows the scheme that is presented by Alonso-Herrero et al. (2009), with some minor modifications (see Relaño et al. 2010, for details).
At first we discuss the accuracy of the trace mask in Sect. 4.1, and study the importance of using a cross-talk correction in Sect. 4.2. Thereafter we compare the outcome of the wavelength calibration of P3D with the outcome created using IRAF instead in Sect. 4.3. In Sect. 4.4 we evaluate the accuracy of the three spectrum extraction methods. Finally, in Sect. 4.5 we compare resulting spectra using the different extraction approaches with observations of planetary nebulae.
4.1 Estimating the accuracy of calculated trace masks
Accurately determined traces, i.e. spectrum center positions, are needed to extract spectra properly (cf. Sect. 4.4). The automatic tracing algorithm of P3D is mostly able to locate all spectra at one wavelength, without any interaction, in the first step. During the spectrum tracing across all wavelengths, of the second step, the accuracy of the result depends on how well the profile center positions are determined.
![]() |
Figure 5: This figure illustrates
the influence of the cross-spectrum smoothing of step 2 of the
tracing algorithm. Four trace residuals, which are calculated from
typical continuum images, are shown for each IFU as a function of
the dispersion axis: a) LARR, b) PPAK, c) SPIRAL, and d) VIRUS-P.
The traces are selected uniformly across the IFU surface. The
black (light gray) solid lines show the respective trace where the
|
Open with DEXTER |
In Fig. 5 we compare calculated traces, which are smoothed across several spectra ( = 11, 21, cf. Sect. 3.1.2), with unsmoothed traces (
= 1). The difference between the two sets are
pixel
in every case, except at the blue and red ends where differences are up
to about 0.5 pixel. All four IFUs show semi-periodic offsets in
the unsmoothed traces, which is due to inaccurate pixel subsampling.
The oscillation period of a trace corresponds to the number of
illuminated pixels of a row on the detector. We illustrate this for one
typical trace of the LARR IFU in Fig. 6a, and for one similarly typical trace of the PPAK IFU in Fig. 6b.
The trace of the LARR IFU extends across about 10.5 pixels on
the cross-dispersion axis, corresponding to 10.5 oscillation
periods, which are seen in the second trace from the top in Fig. 5a.
Likewise, the PPAK trace extends across about 2 pixels, which
equals the number of oscillation periods of the two middle traces in
Fig. 5b. In addition
to offsets of inaccurate pixel subsampling, an additional offset is
caused by the smoothing across several spectra. In this context
the exact value of
seems unimportant as differences are both lower and higher when the
traces of either smoothed mask are compared with the unsmoothed mask.
![]() |
Figure 6: One trace is shown for the LARR IFU in panel a), and for the PPAK IFU in panel b). The position on the cross-dispersion axis is in both cases drawn as a function of the position on the dispersion axis. The solid line shows the smoothed trace and the dash-triple-dotted line the unsmoothed trace. Compare with Figs. 5a and b, cf. Sect. 4.1. |
Open with DEXTER |
Next we compared traces created with P3D and IRAF for the LARR IFU. At first we show four residual traces where the trace mask of P3D was subtracted from the trace mask of IRAF, cf. Fig. 7a. As the figure shows these residuals are always <0.2 pixel, and mostly <0.1 pixel. The residuals of the unsmoothed trace mask show slightly higher variations. We also show residuals using fitted traces (where the line-center positions were fitted together with the intensities when calculating the profiles). In this case the fitted traces differ slightly from both the fixed traces and those of IRAF. Note that the fitted traces do not oscillate across the dispersion axis. In comparison to the outcome of the Gaussian weighting (Fig. 7), this indicates a negligible influence on Gaussian-fitted center positions due to pixel subsampling.
For this test we also fitted the traces of P3D with a fourth order linear polynomial, which we then subtracted from the trace mask of IRAF. We show the result in Fig. 7b. In this case the unsmoothed traces show a better agreement with the traces of IRAF. A comparison between lines of the two panels show that the difference is tiny when the fitted traces are replaced with a polynomial fit. The minimum and maximum values of the residual of all spectra are -0.17 and 0.22 pixel. The mean and standard deviations are -0.043 and 0.044 pixels. In Fig. 9 we show an image of the residuals of all spectra. The discrepancy is, again, greatest at the blue and the red ends.
To compare the calculated center positions with the raw data, we indicate the center positions of each approach in Fig. 8 for two wavelengths. We selected the bluemost part of the second trace from the top in Fig. 7 (lefthand panel), and the reddest part of the topmost trace (righthand panel), since this is where the difference is the largest in the residuals shown. In the former case the fitted center position of P3D lies closer to the position of IRAF, the difference compared with the pre-calculated trace mask position is 0.11 pixel. In the latter the same difference is only 0.012 pixel, while the position of IRAF is more offset. The profile center positions of P3D are more accurate here than those of IRAF.
![]() |
Figure 7: Four trace residuals are shown for a typical continuum image of the LARR IFU as a function of the dispersion axis: a) trace(IRAF)-trace(P3D), b) trace(IRAF)-fit(trace(P3D)); the traces are selected uniformly across the IFU surface. The black (gray) lines show the residuals using the smoothed (unsmoothed) traces of P3D. The dotted line shows residuals of traces that were fitted anew during the profile calculation. To plot all four traces using the same axis, they are offset by half a pixel from each other. The dark gray horizontal lines are guides. For more details see Sect. 4.1. |
Open with DEXTER |
![]() |
Figure 8: Profile center positions across the cross-dispersion axis for IRAF, P3D, and fitted profiles of P3D for two wavelength bins of two different spectra on the detector. The raw data is drawn as a histogram, and the Gaussian fitted profile is shown as a continuous curve. For more details see Sect. 4.1. |
Open with DEXTER |
![]() |
Figure 9: In this image we show the difference between a trace mask that is calculated using P3D and a corresponding trace mask that is calculated using IRAF. For more details see Sect. 4.1. |
Open with DEXTER |
We conclude that the error of (fixed) traces, which are calculated using P3D, should be pixel.
For spectra that are located close to the center of the detector,
this value is probably better. At the blue end of the detector,
for
4000 Å
the accuracy is likely worse because of a lower detector sensitivity.
If high accuracy is required, as may be the case when
cross-talk is present (cf. Sect. 4.4),
the preferred method is to replace the fixed center positions of
the trace mask with fitted positions. It may be difficult,
however, to achieve the highest accuracy of the spectrum
extraction with flexure-affected instruments. Because, spectra might
have moved on the detector during the time between the calibration
exposure and the object exposure. A solution to this problem could
be to re-center the calculated profiles using the object data.
Table 3: Parameters and outcome of our idealized study of the influence of cross-talk.
![]() |
Figure 10: This figure illustrates
effects of fiber-to-fiber cross-talk between neighbor spectra for
typical idealized profiles of four IFUs: a) LARR, b) PPAK, c) SPIRAL (Blue arm), and d) VIRUS-P (bundle 2; bin |
Open with DEXTER |
4.2 Importance of correcting for fiber-to-fiber cross-talk
The same calibration data as is used to calculate the trace mask is
also used to calculate cross-dispersion line profiles, which are used
in the optimal extraction. There is some overlap between spectra on the
cross-dispersion axis for most instruments. The magnitude of the
overlap depends on three factors: the spectrum separation (d),
the width of the profiles, and the intensity of each spectrum.
To estimate the influence of cross-talk, we first calculated line
profiles for different setups of each supported IFU. In this
idealized study, we assumed a Gaussian shape of the profile in every
case. This simplified treatment neglects the effect of extended profile
wings, which come from scattered light, typically at peak intensity, but large FWHM (cf. e.g. Becker 2002).
Using data from different observing runs we first measured both the
variation and the average value of the spectrum width across
the IFU. We present the outcome in Table 3. The initial FWHM of the profiles of each group (
=
)
was taken from Table 2. The calculated average width and its standard deviation are given in Col. 4 (
), and the full range of measured widths in Col. 5 (
).
The width varies across the detector for all IFSs, in most cases
across both axes. This variation should be kept in mind when
interpreting the percentages we present next.
For each IFU configuration, we then calculated a set of 3-4 line profiles using the spectrum width
and the spectrum separation d. We integrated the flux for one (reference) profile across an aperture of pre-defined width
(
=
,
see Table 2) and calculated the fraction of the flux that fell outside the aperture (
).
Additionally, we calculated the fraction of increased flux inside the
aperture, because of the two neighbor profiles, in order to
estimate the cross-talk contribution. Doing this we assumed that the
intensity of the righthand profile is 1, 10, and 100 times
higher (
,
,
)
than the reference profile. The outcome for each IFU setup is given in Table 3. We also illustrate the profiles of four setups in Fig. 10.
This study reveals several important results. For the LARR IFU it is evident that a cross-talk correction is unnecessary. Even if there is a strong intensity gradient across the IFU, with an intensity ratio of 100 between two neighboring spectra, the amount of cross-talk is only about 1%. The amount of flux outside the aperture of the reference spectrum of the PPAK/IFU is negligible (0.44%). Since fibers are more tightly packed, however, the amount of cross-talk is significant. If intensity gradients are moderate (and ratios are <10), the intensity of the reference profile is increased by about 12% thanks to cross-talk. With the remaining IFU configurations, the amount of flux outside the aperture is 1-5%, while the cross-talk contribution to the reference profile increases to 7-28% for moderate overlap and to 67-260% for strongly overlapping profiles (intensity ratio of 100). With the exception of the LARR IFU, it is clear that it is necessary to correct for cross-talk to achieve any level of accuracy in integrated fluxes of weaker regions of the object. Even for the LARR IFU an optimal extraction could be advantageous if it is important to extract weak lines accurately (cf. Sect. 4.5).
![]() |
Figure 11:
This figure shows histograms and residuals of fitted emission lines
from two extracted and wavelength-calibrated arc images of
LARR data. In the top (bottom) row we show plots of Hg
|
Open with DEXTER |
Allington-Smith & Content (1998) argue that a cross-talk correction is unneccessary when neighboring spectra on the detector correspond to well-sampled neighboring positions on the sky, as is the case with the LARR and the SPIRAL IFUs (and also the VIMOS and the FLAMES-ARGUS IFUs). This argument holds particularly well whenever precision spatial sampling is not the driver for a given application, such as when using adaptive binning with SAURON observations (Cappellari & Copin 2003). However, owing to properties of a datacube, which can be seen as a stack of hundreds of images, IFS with proper spatial sampling has the potential of delivering extremely accurate astrometry with milli-arcsec centroiding precision for point sources from ground-based observations (Roth et al. 2004). For these applications, where an accurate definition of the point-spread-function is essential, cross-talk corrections are very important (ibid. Fig. 4). S10 (see their Fig. 7), for the SPIRAL IFU, also find unwanted artefacts in their extracted spectra when they do not apply a cross-talk correction.
While spectra of most IFUs are arranged next to each other on both the sky and on the detector, this is not the case for the PPAK IFU. Nearby spectra on the sky are on PPAK placed at different locations on the CCD. When seeing prevents extreme intensity gradients for IFUs with densely packed elements, intensity ratios of as much as 1000 are easily achieved with PPAK data. Such as observing the center of a planetary nebula and its thousand times fainter halo. In this case it is necessary to use a two-dimensional approach of data reduction so as to extract the weak component properly (cf. Bolton & Schlegel 2010).
4.3 Comparing the wavelength calibration correction between P3D and IRAF
Here we compare how well the wavelength calibration correction of P3D matches a correction that is carried out with IRAF instead. Using data of the LARR IFU, we calculated three dispersion masks, which we then applied to the extracted spectra of the respective arc image. We used the same line list with both tools. With P3D we used a fifth order linear polynomial, and with IRAF a fifth order Legendre polynomial. The observations were done using the V300 grating of PMAS, where the dispersion is 1.67 Å/pixel.
We evaluated the accuracy of either correction by fitting a
Gaussian profile to all spectra, using several arc lines in the
extracted and wavelength-calibrated images. Specifically we fitted two
mercury lines with the rest wavelengths = 4358.328, 5460.735 Å and two neon lines with
=
5975.534, 6598.953 Å. The standard deviation of the center
positions were found to be similar for the two tools and to vary
between 0.06-0.31 Å (IRAF) and 0.09-0.13 Å (P3D) for the four lines and all spectra of the three images. In Fig. 11 we plot histograms of the fitted wavelengths for one of the dispersion masks and Hg
and Ne
.
For the mercury line (Figs. 11a and b), the respective tool provides the average (and standard deviation) value of 5460.71 (0.15, IRAF) and 5460.69 (0.11, P3D). The results are evidently very similar in this case. For the neon line the corresponding values are 5975.43 (0.07, IRAF) and 5975.60 (0.12, P3D). In this case P3D shows a larger scatter of the values. The residual plots (Figs. 11c and f) confirm that the standard deviation of values of IRAF is systematically smaller than with P3D, but differences are small. The wavelength-calibration correction of P3D can probably be further improved if additional care is taken to enhance the resampling algorithm to a common wavelength.
![]() |
Figure 12:
This figure shows measured-to-modeled flux ratios for the PPAK IFU (using
|
Open with DEXTER |
The accuracy that can be achieved with the wavelength calibration depends on several factors, four of which are: the spectral resolution, the number of entries in the line mask and the accuracy of the center pixel positions, the fitting function (and its order if it is a polynomial), and properties of the final interpolation to a common wavelength for all spectra. We have the following four recommendations regarding the preparation of a line list: lines must be isolated, lines with high S/N are preferred, lines that are (close to) saturated should be avoided, and lines must be distributed across the full spectral range of interest.
4.4 Accuracy of spectra extracted using the three different methods
To measure the accuracy of the spectrum extraction, we tested the three extraction methods with idealized simulated data. With this motive we assumed that the line profiles are Gaussian. All spectra are perfectly aligned with the dispersion axis and the intensity is invariant with wavelength. Hereby we only model one wavelength bin.
Setting up our simulation we first defined a set of ten
Gaussian profiles for each of the four IFUs. We used the average
instrument-specific profile widths,
,
which we calculated in Sect. 4.2 (see Table 3). The Gaussian profiles are, moreover, separated by
pixels (see Table 2).
We set the intensities of all but two profiles to the same value;
the third and sixth profile intensities were set ten and hundred
times higher. Thereafter, the profiles were scaled to a pre-defined
value of the signal-to-noise (S/N) of the weaker
profiles and summed up to create one spectrum. We refer to the scaled
intensities as modeled intensities below. For the noise model, we
used a Poissonian noise distribution and the instrument-specific
readout noise. In all subsequent measurements, we used the
intensity of the ninth, second, and fifth profiles. These profiles
correspond to measuring a weaker line that lies next to a line that
is 1, 10, and 100 times as strong.
To evaluate the simulation we used an approach that is similar to that of S10. For each IFU setup, we fitted three different sets of Gaussian profiles to the simulated data. For the first set we used the already known center positions and profile widths, and scaled the input intensities to achieve a S/N of 2-300. For the second set, we used the known profile width and set the input intensities to correspond to S/N = 100, and introduced an error to the pre-determined profile positions of 0.0-0.7 pixel. In the third set we used the known profile center position and again used S/N = 100. This time we introduced an error into the profile width, which is used in the fitting, of -0.4-0.6 pixel. A negative error corresponds to a narrower profile and a positive error to a wider profile. For every configuration we fitted the profiles 100 times using the P3D-routine of the respective extraction method, using as many realizations of the noise model, and saved the average intensities.
In the following text we refer to the modified optimal extraction method as MOX (Sect. 3.2.2). We, likewise, refer to the multi-profile deconvolution optimal extraction method as MPD (Sect. 3.2.3).
We begin our analysis with the PPAK IFU. In Fig. 12a
we show the result of the high-intensity simulations where we
introduced an error to the pre-determined center positions of the three
profiles. Using the aperture extraction it is seen that the extracted
flux, due to cross-talk, always is higher than the model input
intensity. The values at error 0.0 agree with the values in
Sect. 4.2 (see Table 3). We did not use discrete pixels in that study. The measured flux is % higher
than the model flux with the highest intensity ratio. If there are
strong intensity gradients in the data and it is important to measure
weak regions accurately, it is not recommended to use the aperture
extraction method. Using MOX the accuracy is higher than with the
aperture extraction. Although with MOX the error increases rapidly with
the error of the pre-determined center position for the two profiles,
which lie next to the intenser profiles. MPD shows the best
performance of all methods at small errors. Although, for larger
errors (
px)
it is outperformed by the other two methods when the intensity ratio is
near unity. With MPD it is enough to keep the profile center error
smaller than 0.2 pixel in order to achieve an intensity error that
is smaller than 5% for all intensity ratios. Using MOX the
corresponding error of the pre-determined center position must,
in this case, be smaller than about 0.03 pixel.
We study the influence of an accurate pre-determined profile FWHM in Fig. 12b.
Both optimal extraction methods show a similar dependence for positive
errors. MPD is seen to give a slightly higher accuracy at
reasonably low positive errors ( pixel).
When the pre-determined FWHM is too narrow, and the error consequently
is negative, MPD shows smaller errors in the flux than for
positive FWHM errors. With MOX the resulting flux errors are
similar, as for positive FWHM errors. To calculate
fluxes, which errors are smaller than 5%, it is necessary to
keep the FWHM error smaller than 0.05 pixel (0.2 pixel) with
MOX (MPD).
The dependence of the resulting flux error on the S/N is shown in Fig. 12c.
The line of the factor-100 intensity ratio is offset by -1.0 on
the ordinate. It is seen that values change little with the S/N, with the exception of low values (S/N 4). For S/N
30,
and the highest intensity ratio (100), the resulting fluxes,
which are calculated using MOX, are about 3% lower than
corresponding values, which are instead calculated using MPD.
![]() |
Figure 13:
Figure 12 is repeated for LARR (top row, panels a-c;
|
Open with DEXTER |
In Sect. 4.2 we showed that there is no need to correct LARR data for cross-talk even for high inter-profile intensity ratios, as spectra are separated very well. To provide a complete study we, nevertheless, test the outcome when such a correction is made anyways.
Figure 13a shows
that MOX gives accurate results for small center position errors and
higher intensity gradients. MOX gives unreliable fluxes when the
intensity ratio is unity, which are
of the true flux for small errors, see Fig. 13b.
The offset lines (at flux ratio 1.4) show the calculated
fluxes using MOX, without a cross-talk correction, i.e., for
regular optimal extraction. The lines illustrate that flux errors are
small, even at relatively high errors of the center positions and the
FWHM. It does not make any sense to use MPD or MOX with cross-talk
correction with LARR data. In this case noise in the profile
wings increase the errors of the measured flux, instead of
decreasing them.
With the SPIRAL IFU and the VIRUS-P IFU, see Figs. 13d-i,
we see a similar behavior as for the PPAK IFU. With these IFUs and
MPD it is necessary to keep errors of the pre-determined center
positions pixel (SPIRAL) and
pixel (VIRUS-P) in order to delimit flux errors to 5%. With MOX the center error should be
pixel
(SPIRAL) and <0.03 pixel (VIRUS-P). The required precision of
center positions is very high for SPIRAL, with both methods, which is
why it seems unrealistic to achieve this high accuracy in fluxes.
To delimit errors we recommend to fit the spectrum center
positions, along with the intensities, when calculating line profiles
for this IFU (cf. Sect. 4.1). Both methods, moreover, require an accuracy in the profile width of
pixel in order to constrain the measured flux error to 5%. The S/N-test for VIRUS-P (Fig. 13i) shows that the error of MPD is about half that of MOX for S/N
10. For the SPIRAL IFU,
significant flux errors (
)
are only introduced, as a function of S/N, with MOX (Fig. 13f).
We conclude that MPD is the preferred method of spectrum extraction
whenever cross-talk is present. The cases where MOX was found to
outperform MPD can probably be explained by the smaller extraction
width that is used with MOX. Compare
with
,
which were both defined in Sect. 3.2.
Regardless of the chosen method of optimal extraction, it is
always important to minimize errors of the pre-determined center
positions and FWHM. Errors of measured fluxes grow fast if there are
significant intensity gradients in the data. We found that MOX is
highly sensitive to accurate spectrum center positions
(i.e. traces). MOX also introduces some error to data with
low S/N. Although with LARR data, which is free
of cross-talk, MOX is the preferred method (because of the way the
extraction width of both methods is defined within P3D),
without the optional cross-talk correction. Finally, with the exception
of LARR data, aperture extraction always introduces significant
errors to calculated fluxes of data, which contains any spatial
intensity gradients.
![]() |
Figure 14: Comparison of spectra, which are calculated using the aperture extraction, MOX, and MPD. In panel a) we show spectra of LARR and in panel b) spectra of PPAK. The flux is shown as a function of wavelength - the two panels are plotted for different wavelength ranges. Solid black (gray) lines show spectra using MOX (aperture extraction). In panel b) we used MPD to calculate the spectrum that is drawn with a dotted line. In panel a) the two lower (upper) spectra use the left (right) y-axis. Error bars are shown for parts of the spectra in both panels. Wavelengths of a few emission lines (cosmic ray hits) are indicated with a vertical line and the name of the line (CR). For more details see Sect. 4.5. |
Open with DEXTER |
4.5 Comparing resulting emission line spectra of planetary nebulae using the three different extraction methods
Finally, we compare the outcome of spectra, which were extracted using the different methods. In Fig. 14
we show wavelength-calibrated and flat-fielded spectra of data of LARR
and PPAK from the central regions of two planetary nebulae:
NGC 2392 for PPAK and M 2-2 for LARR, whose outcome is
presented in Sandin et al. (2008).
In the first case we reduced the LARR data using MOX (without
correcting for cross-talk) and aperture extraction. As Fig. 14a
shows, the spectrum of a single spatial element is less noisy when
MOX is used instead of aperture extraction. The error bars of MOX are
half as high than with aperture extraction. In the spectrum that
is averaged over 12 spatial elements, we see that the continuum
level of the MOX-spectrum is lower than that of the aperture-extracted
spectrum (by about 55%). In this emission line object,
scattered light becomes significant for the weak continuum that is
dominated by readout noise (compare the gray and black lines in
Fig. 15a). Using aperture
extraction more of the scattered light is included in the resulting
spectrum, and weak emission lines are underestimated. Both methods
calculate the same intensity for H.
We show Fig. 14b,
to illustrate the result of cross-talk in PPAK data. In this
case MOX and MPD give very similar solutions, except at the
location of the cosmic ray hit, at
6474 Å.
The flux at the wavelength of the cosmic ray hit was removed by
the MOX-algorithm. That spectrum lies in a weak part of the central
nebula on the sky, but on the CCD the neighbor spectrum lies in a more
intense part of the nebula; compare apertures 9 and 8
in Fig. 15b. Using
aperture extraction flux of the neighbor spectrum falls within the
aperture of the weak spectrum, and increases the flux of H
significantly above its true value. In this case we attribute the
difference in the continuum levels of the aperture and MOX/MPD methods
to both scattered light and cross-talk. The continuum-region error bars
of MOX are on average 0.88 times as high as the aperture
extraction error bars.
5 Conclusions
We have presented a new and general data-reduction tool for fiber-fed IFUs. Our goal has been to write a user-friendly tool that works stably with different sets of instruments and data. In comparison to similar tools, the strong advantage of P3D is that it can find and trace all spectra on the detector, mostly without any user interaction at all. Using the methods of optimal spectrum extraction data of all IFUs can also be corrected for cross-talk, which arises from the overlapping spectra on the detector. Since the same procedures are used with all implemented IFUs, it is, moreover, a straightforward task to compare the outcome of different observations. Although P3D is based on the proprietary Interactive Data Language (IDL), its use requires no IDL license. All components of P3D work with all platforms supported by IDL. The program code can be downloaded from the project web site at http://p3d.sourceforge.net.
To validate the program code, we tested the different parts using both simulated data and corresponding outcome of IRAF. We found that P3D produces results comparable to IRAF. For all IFUs, with the exception of the lens array of PMAS, P3D is able to extract more accurate values than IRAF since P3D can correct for cross-talk.
![]() |
Figure 15:
Parts of the bias-subtracted raw data used to extract the spectra of Fig. 14. The counts are plotted for one wavelength across the cross-dispersion axis for data of LARR, panel a), and PPAK, panel b). In panel a) the group of profiles drawn in gray is shown for the center wavelength of H |
Open with DEXTER |
Although P3D has so far been configured for four IFUs, it is a straightforward task to extend it to work with additional instruments. If the new IFU is similar to the already implemented ones it is just a matter of setting up another set of instrument-specific parameters. The same concerns the level of functionality. P3D can with relatively little effort be extended to also handle, for example, flux calibration, correction for differential atmospheric refraction (for those IFUs where it makes sense), removal of cosmic rays in individual images, and sky subtraction, and account for scattered light. Most parts of P3D are, moreover, fast and only require a few seconds to execute on a typical workstation. The calculation of line profiles and the optimal extraction algorithm, however, are more computationally intensive and require a few minutes. The code execution time could in this case be shortened by moving the relevant parts of the IDL-code to compiled (and dynamically loaded) C-code. If, and when, these and other improvements are implemented depends on the need and the interest of the community.
AcknowledgementsDuring a large part of the development time, this project was supported by the grants BMBF 03Z2A51 and BMBF 05A08BA1. A.M.-I. was supported by the Spanish Ministry of Science and Innovation (MICINN) under the program ``Specialization in International Organisms'' (ES2006-0003). We thank L. Cairós for her careful testing of the dispersion mask tool and for providing us with test data sets for bundle 1 of VIRUS-P. We thank O. Streicher both for his careful testing of all parts of P3D and for having written tools for the project web site, which automatically extracts and presents the routine documentation. Q. Parker and M. Relaño are thanked for providing us with test data sets for the LARR and SPIRAL IFUs. We, furthermore, thank A. Zwanzig for writing extensive installation instructions. We finally thank J. Adams and R. Sharp for providing us with instrument-specific information, which allowed us to configure P3D for VIRUS-P and SPIRAL, respectively.
References
- Allington-Smith, J., & Content, R. 1998, PASP, 110, 1216 [NASA ADS] [CrossRef] [Google Scholar]
- Allington-Smith, J., Murray, G., Content, R., et al. 2002, PASP, 114, 892 [NASA ADS] [CrossRef] [Google Scholar]
- Alonso-Herrero, A., García-Marín, M., Monreal-Ibero, A., et al. 2009, A&A, 506, 1541 [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
- Arribas, S., Carter, D., Cavaller, L., et al. 1998, in Proc. SPIE 3355, ed. S. D'Odorico, 821 [Google Scholar]
- Arribas, S., Mediavilla, E., García-Lorenzo, B., del Burgo, C., & Fuensalida, J. J. 1999, A&AS, 136, 189 [Google Scholar]
- Avila, G., Guinouard, I., Jocou, L., Guillon, F., & Balsamo, F. 2003, in Proc. SPIE 4841, ed. M. Iye, & A. F. M. Moorwood, 997 [Google Scholar]
- Becker, T. 2002, Ph.D. Thesis, Univ. Potsdam [Google Scholar]
- Bershady, M. A. 2009, in 3D Spectroscopy in Astronomy, XVII Canary Island Winter School of Astrophysics, ed. E. Mediavilla, S. Arribas, M. Roth, J. Cepa-Nogue, & F. Sánchez (CUP) [Google Scholar]
- Blecha, A., Cayatte, V., North, P., Royer, F., & Simond, G. 2000, in Proc. SPIE 4008, ed. M. Iye, & A. F. Moorwood, 467 [Google Scholar]
- Bolton, A. S., & Burles, S. 2007, New J. Phys., 9, 443 [NASA ADS] [CrossRef] [Google Scholar]
- Bolton, A. S., & Schlegel, D. J. 2010, PASP, 122, 248 [NASA ADS] [Google Scholar]
- Cappellari, M., & Copin, Y. 2003, MNRAS, 342, 345 [NASA ADS] [CrossRef] [Google Scholar]
- Fabrika, S., Sholukhova, O., Becker, T., et al. 2005, A&A, 437, 217 [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
- Filippenko, A. V. 1982, PASP, 94, 715 [NASA ADS] [CrossRef] [Google Scholar]
- Hill, G. J., MacQueen, P. J., Smith, M. P., et al. 2008, in Proc. SPIE, 7014, 231 [Google Scholar]
- Horne, K. 1986, PASP, 98, 609 (H86) [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
- Howell, S. B. 2006, Handbook of CCD Astronomy, 2nd edn., Cambridge Observing Handbooks for Research Astronomers (CUP) [Google Scholar]
- Hunt, A., & Thomas, D. 1999, The Pragmatic Programmer: From Journeyman to Master (Addison-Wesley Professional) [Google Scholar]
- Kelz, A., Verheijen, M. A. W., Roth, M. M., et al. 2006, PASP, 118, 129 [NASA ADS] [CrossRef] [Google Scholar]
- Kissler-Patig, M., Copin, Y., Ferruit, P., Pécontal-Rousset, A., & Roth, M. M. 2004, Astron. Nachr., 325, 159 [NASA ADS] [CrossRef] [Google Scholar]
- LeFèvre, O., Saisse, M., Mancini, D., et al. 2003, in Proc. SPIE 4841, ed. M. Iye, & A. F. M. Moorwood, 1670 [Google Scholar]
- Lehmann, I., Becker, T., Fabrika, S., et al. 2005, A&A, 431, 847 [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
- Markwardt, C. B. 2009, in Astronomical Data Analysis Software and Systems XVIII, ed. D. Bohlender, P. Dowler, & D. Durand, ASP Conf. Ser., 411, 251 [Google Scholar]
- Monreal-Ibero, A., Roth, M. M., Schönberner, D., Steffen, M., & Böhm, P. 2005, ApJ, 628, L139 [NASA ADS] [CrossRef] [Google Scholar]
- Pasquini, L., Avila, G., Allaert, E., et al. 2000, in Proc. SPIE 4008, ed. M. Iye, & A. F. Moorwood, 129 [Google Scholar]
- Pych, W. 2004, PASP, 116, 148 [NASA ADS] [CrossRef] [Google Scholar]
- Relaño, M., Monreal-Ibero, A., Vílchez, J. M., & Kennicutt, R. C. 2010, MNRAS, 402, 1635 [NASA ADS] [CrossRef] [Google Scholar]
- Roth, M. M., Becker, T., Kelz, A., & Schmoll, J. 2004, ApJ, 603, 531 [NASA ADS] [CrossRef] [Google Scholar]
- Roth, M. M., Kelz, A., Fechner, T., et al. 2005, PASP, 117, 620 [NASA ADS] [CrossRef] [Google Scholar]
- Sánchez, S. F. 2006, Astron. Nachr., 327, 850 (S06) [NASA ADS] [CrossRef] [Google Scholar]
- Sandin, C., Schönberner, D., Roth, M. M., et al. 2008, A&A, 486, 545 [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
- Schmoll, J., Dodsworth, G. N., Content, R., & Allington-Smith, J. R. 2004, in Proc. SPIE 5492, ed. A. F. M. Moorwood, & M. Iye, 624 [Google Scholar]
- Scodeggio, M., Franzetti, P., Garilli, B., et al. 2005, PASP, 117, 1284 [NASA ADS] [CrossRef] [Google Scholar]
- Sharp, R., & Birchall, M. N. 2010, PASA, 27, 91 (S10) [NASA ADS] [CrossRef] [Google Scholar]
- Sharp, R., Saunders, W., Smith, G., et al. 2006, in Proc. SPIE, 6269, 14 [Google Scholar]
- Smith, G. A., Saunders, W., Bridges, T., et al. 2004, in Proc. SPIE 5492, ed. A. F. M. Moorwood, & M. Iye, 410 [Google Scholar]
- Turner, J. E. H. 2006, New Astron. Rev., 50, 392 [NASA ADS] [CrossRef] [Google Scholar]
- van Dokkum, P. G. 2001, PASP, 113, 1420 [NASA ADS] [CrossRef] [Google Scholar]
- Villar-Martín, M., Sánchez, S. F., De Breuck, C., et al. 2006, MNRAS, 366, L1 [NASA ADS] [CrossRef] [Google Scholar]
Footnotes
- ... spectrographs
- Based in part on observations collected at the Centro Astronómico Hispano Alemán (CAHA), operated jointly by the Max-Planck Institut für Astronomie and the Instituto de Astrofísica de Andalucía (CSIC).
- ...
- Time-stamped version of the code is only available in electronic form at the CDS via anonymous ftp to cdsarc.u-strasbg.fr (130.79.128.5) or via http://cdsweb.u-strasbg.fr/cgi-bin/qcat?J/A+A/515/A35
- ... IRAF
- The Image Reduction and Analysis Facility IRAF is distributed by the National Optical Astronomy Observatories which is operated by the association of Universities for Research in Astronomy, Inc. under cooperative agreement with the National Science Foundation.
- ... Language
- http://www.ittvis.com
- ... NASA
- http://idlastro.gsfc.nasa.gov
- ...Markwardt (2009)
- http://purl.com/net/mpfit
- ... IDL
- P3D uses widgets abundantly and can therefore not be used with GDL (cf. http://gnudatalanguage.sourceforge.net).
- ... sky
- There are several ways to subtract the contribution of sky emission lines from object data. For the initial release of P3D we chose not to include any sky subtraction as it is difficult to define a general procedure for it.
- ... 3620-5910 Å
- This wavelength range can, if necessary, easily be changed by providing P3D with alternative lower and upper values.
All Tables
Table 1: A list of fiber-fed IFUs and their respective data-reduction pipelines.
Table 2: General and instrument-specific parameters of the data-reduction algorithms of P3D.
Table 3: Parameters and outcome of our idealized study of the influence of cross-talk.
All Figures
![]() |
Figure 1: This figure shows two
adjacent PPAK raw data sections highlighting one group of
32 spectra. We show continuum lamp data on the lefthand side and
object data, about H |
Open with DEXTER | |
In the text |
![]() |
Figure 2:
In this figure we illustrate how adjacent spectra are separated for four IFUs: a) PMAS-LARR (using bin |
Open with DEXTER | |
In the text |
![]() |
Figure 3:
The result of step 1 of the tracing algorithm for continuum lamp data of the PPAK IFU, cf. Sect. 3.1.1. In this case the used CCD binning setup was
|
Open with DEXTER | |
In the text |
![]() |
Figure 4: This figure shows three extracted arc frames of: a) LARR, b) LARR after the dispersion solution was applied, and c) VIRUS-P. Colors are modified to enhance various features. In each panel the vertical line with a disk indicates a preset reference spectrum and wavelength bin. In panel a) the arrow points at the merging point of a shorter (1s; left part) and a longer (10s; right part) exposure. This straight line is curved in panel b) (left arrow). In panel b) the right arrow points at an emission line that was outside the calibration range. In panel c) the left arrow indicates the curved shape of a line in the line mask, after the line geometry has been calculated. The two right arrows indicate a mismatch between the line mask and the data for redder wavelengths. For more details see Sect. 3.3. |
Open with DEXTER | |
In the text |
![]() |
Figure 5: This figure illustrates
the influence of the cross-spectrum smoothing of step 2 of the
tracing algorithm. Four trace residuals, which are calculated from
typical continuum images, are shown for each IFU as a function of
the dispersion axis: a) LARR, b) PPAK, c) SPIRAL, and d) VIRUS-P.
The traces are selected uniformly across the IFU surface. The
black (light gray) solid lines show the respective trace where the
|
Open with DEXTER | |
In the text |
![]() |
Figure 6: One trace is shown for the LARR IFU in panel a), and for the PPAK IFU in panel b). The position on the cross-dispersion axis is in both cases drawn as a function of the position on the dispersion axis. The solid line shows the smoothed trace and the dash-triple-dotted line the unsmoothed trace. Compare with Figs. 5a and b, cf. Sect. 4.1. |
Open with DEXTER | |
In the text |
![]() |
Figure 7: Four trace residuals are shown for a typical continuum image of the LARR IFU as a function of the dispersion axis: a) trace(IRAF)-trace(P3D), b) trace(IRAF)-fit(trace(P3D)); the traces are selected uniformly across the IFU surface. The black (gray) lines show the residuals using the smoothed (unsmoothed) traces of P3D. The dotted line shows residuals of traces that were fitted anew during the profile calculation. To plot all four traces using the same axis, they are offset by half a pixel from each other. The dark gray horizontal lines are guides. For more details see Sect. 4.1. |
Open with DEXTER | |
In the text |
![]() |
Figure 8: Profile center positions across the cross-dispersion axis for IRAF, P3D, and fitted profiles of P3D for two wavelength bins of two different spectra on the detector. The raw data is drawn as a histogram, and the Gaussian fitted profile is shown as a continuous curve. For more details see Sect. 4.1. |
Open with DEXTER | |
In the text |
![]() |
Figure 9: In this image we show the difference between a trace mask that is calculated using P3D and a corresponding trace mask that is calculated using IRAF. For more details see Sect. 4.1. |
Open with DEXTER | |
In the text |
![]() |
Figure 10: This figure illustrates
effects of fiber-to-fiber cross-talk between neighbor spectra for
typical idealized profiles of four IFUs: a) LARR, b) PPAK, c) SPIRAL (Blue arm), and d) VIRUS-P (bundle 2; bin |
Open with DEXTER | |
In the text |
![]() |
Figure 11:
This figure shows histograms and residuals of fitted emission lines
from two extracted and wavelength-calibrated arc images of
LARR data. In the top (bottom) row we show plots of Hg
|
Open with DEXTER | |
In the text |
![]() |
Figure 12:
This figure shows measured-to-modeled flux ratios for the PPAK IFU (using
|
Open with DEXTER | |
In the text |
![]() |
Figure 13:
Figure 12 is repeated for LARR (top row, panels a-c;
|
Open with DEXTER | |
In the text |
![]() |
Figure 14: Comparison of spectra, which are calculated using the aperture extraction, MOX, and MPD. In panel a) we show spectra of LARR and in panel b) spectra of PPAK. The flux is shown as a function of wavelength - the two panels are plotted for different wavelength ranges. Solid black (gray) lines show spectra using MOX (aperture extraction). In panel b) we used MPD to calculate the spectrum that is drawn with a dotted line. In panel a) the two lower (upper) spectra use the left (right) y-axis. Error bars are shown for parts of the spectra in both panels. Wavelengths of a few emission lines (cosmic ray hits) are indicated with a vertical line and the name of the line (CR). For more details see Sect. 4.5. |
Open with DEXTER | |
In the text |
![]() |
Figure 15:
Parts of the bias-subtracted raw data used to extract the spectra of Fig. 14. The counts are plotted for one wavelength across the cross-dispersion axis for data of LARR, panel a), and PPAK, panel b). In panel a) the group of profiles drawn in gray is shown for the center wavelength of H |
Open with DEXTER | |
In the text |
Copyright ESO 2010
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.