3 Mapping order locations

In general echelle order curvature and location on the detector can either be assumed a priori or determined empirically from the image being reduced. Assumed order maps can be calculated from an optical model or measured empirically from a reference image. Alignment of the assumed map with actual order locations may require an offset. Although some echelle spectrographs may be stable enough to rely on an assumed order map, we believe a general echelle reduction package should determine order curvature and location empirically, whenever possible.

Empirical order maps may be determined interactively or automatically. Automated procedures are preferable because they are reproducible and allow batch processing of many spectra, though the robustness of human interaction must then be captured in an automated algorithm. Automated procedures must also evaluate whether order mapping has been successful, halting or reverting to a default order map if a specific image cannot be mapped, for example when there is no signal.

The new order location algorithm presented here consists of four steps: selection of pixels that might be in spectral orders, clustering analysis of these selected pixels, merging and rejection of clusters, and fitting of merged clusters. The procedure is sufficiently robust to use on every science image, but UVES is stable enough that order locations need only be determined once, for example using the mean flat. A special order definition image may be needed when an image slicer is used as the entrance slit.

Recall that after forcing a canonical orientation, spectral orders are aligned approximately along image rows. Selection of pixels that might be in spectral orders is done by first smoothing each column and then selecting pixels above the median of the difference between the original and the smoothed column:

$$a_{x,y} \mbox{\ is selected if\ } a_{x,y}>\overline{a}_{x,y}+\textrm{MEDIAN}(a_{x,y}-\overline{a}_{x,y}).$$ (1)

The smoothing filter can be tuned depending on order separation and contrast. Figure 1 shows an example of pixels selected by the criterion in Eq. (1).

The x and y indices of selected pixels are stored and used in the clustering analysis, which associates connected groups of pixels. One can think of this procedure as coloring selected pixels so that all neighboring pixels (with x and y that differ by at most 1) have the same color. This is a non-trivial problem because some initially detached clusters may ultimately need to be merged, as illustrated in Fig. 1.

Figure 3: Pixel sampling of a spectral order is usually optimized to use detector area efficiently, while retaining adequate sampling. This minimal sampling results in visible "interference'' between periodic pixel spacing and quasi-periodic spacing of columns where orders cross from one detector row to the next. This is seen as "waves'' in the maximum amplitude along an order. The large depressions in this fragment of the UVES CS 22892-052 spectrum are absorption lines.

Coloring begins by creating a vector to store the color of each selected pixel in x and y. Neighbors in the same row are easily detected by sorting x and y so that y increases monotonically, while x scans the row corresponding to each value of y. In order to make the procedure as efficient in the vertical direction, we also sort x and y so that x increases monotonically, while y scans columns. We use a lookup array that relates both sets of sorted coordinates to assign a common color to neighboring pixels, thereby forming a cluster. Spurious clusters may persist in areas of very low signal. After the coloring step, clusters with fewer pixels than some tunable threshold are deselected, causing them to be ignored in the final fitting step.

Orders are often truncated by the top or bottom edge of the detector, potentially hampering attempts to fit order location. To avoid this problem, columns with cluster pixels in the first or last row of the detector are ignored, when fitting the remaining pixels in a cluster (see Fig. 2). This prepares for accurate extraction of partial orders.

Figure 4: A segment of a flat field order (top) and the corresponding model image (bottom) reconstructed from the decomposed spectrum and oversampled (M=10) spatial profile. The flat field image was obtained with the ESO UVES spectrometer using an image slicer. Note the bad row and the low sensitivity pixel (a trap) near the right edge of the observed order. In the model reconstruction the bad row has been completely removed with the help of a mask, and the trap is automatically removed by the decomposition algorithm.

Even after clustering analysis, a single order may be partitioned into multiple clusters due to detector defects or deep absorption features. Polynomial fits are useful for deciding whether to merge or discard clusters. For each cluster, we fit a polynomial of tunable order to pixel y values, as a function of x, and then extend the fit to cover all columns. Extended fits for pairs of consecutive clusters are compared to identify which pair has fits that are most coincident for x values present in the later cluster. As a metric of coincidence, we use the number of columns in which both fits disagree by less than a few pixels.

Figure 5: A segment of a stellar spectrum (HD 217522) obtained with an image slicer (upper left) and the corresponding decomposition into an oversampled (M=10) spatial profile (upper right) and the spectrum (lower right). The reconstructed model image (Eq. (4)) is shown in the lower left. The upper-right panel also shows data for every column, scaled and offset according to the model. The only significant outlier corresponds to a cosmic ray feature in the observation which left no trace in the extracted spectrum. The regions used to estimate scattered light are marked in the upper-right panel.

Figure 6: Flat field normalization using spatial profiles. Panels from the top: spatial profile for individual swaths, the derived order shape function, a segment of the mean flat field image containing part of a single order, and the same segment after normalization using spatial profiles.

In automatic mode, the most coincident pair of clusters is merged if the two fits coincide over at least 95% of the x values present in the later cluster. Otherwise, the smaller of the two clusters is ignored. After merging or discarding a cluster, coincidence is re-assessed based on new polynomial fits. Merging halts when no clusters coincide. This procedure must converge in less than $N_{\rm clusters} - N_{\rm orders}$iterations but in practice the detector properties and layout of the focal plane may allow more rapid convergence.

If the automated merging algorithm fails, manual intervention is required. The only situation we found requiring such intervention was the case of a bad row crossing multiple spectral orders (Fig. 2). In this case, fits to partial clusters created by the bad row may result erroneously in more significant coincidence with an adjacent order.

For each order, our clustering analysis yields a polynomial description of order location, an uncertainty estimate for the fitted polynomial, and the beginning and ending columns to use during spectrum extraction. In addition, we attempt to identify the echelle order number of the first and last order in the image, based on the observed change in order spacing. For a grating cross-disperser, the grating equation for the echelle (assuming that each detector column corresponds to a constant reflection angle) gives:

$$\Delta y\cdot n\cdot (n + 1) = \mbox{const.}$$ (2)

where $\Delta y$ is the separation between echelle orders n and n+1along a fiducial column on the detector. An analogous relationship can be defined for prism cross-dispersers. In principle, three consecutive orders are sufficient to determine n, even without knowledge of the constant. In practice, echelle spectrographs operate at very high n, making Eq. (2) rather similar for adjacent order pairs. Nonetheless, echelle order identification is usually reliable for the large number of orders typically present in many echelle spectrographs.