Appendix D: Covariances method for error and variability analysis

The degree of variability in a particular spectral point over a series of spectra can be expressed by the variance in that spectral point. The variances are often used to derive error estimates on the values of spectral points, and to detect intrinsic spectral variability. Usually two adjacent spectral points do not behave independently from eachother. Ignoring the covariability of adjacent spectral points can lead to serious under-estimation of the error on a quantity derived by integration along the spectrum. A correct error analysis therefore involves the calculation of covariances. Better than variances, covariances are powerful in detecting variability on an intermediate spectral scale - i.e. exceeding the instrumental profile but considerably smaller than the total spectral range. Covariances have also been used successfully in proving the reality of spectral features with low signal-to-noise (van Loon et al. 1996).

For a set of N spectra, each consisting of a number of spectral values x, the covariance of points a & b is:

$$\sigma_{\rm ab}^2 = \frac{ N \sum_{k=1}^N x_{k{\rm a}}x_{k{\r... ... x_{k{\rm a}} ) (\sum_{k=1}^N x_{k{\rm b}} ) }{ N (N-1) }\cdot$$ (D.1)

It measures how much and how coherently the values of the two points vary from one spectrum to another. For $\rm a=b$ the covariance reduces to the variance. Integrating the spectral values x over a spectral range (A,B) in spectrum k yields:

$$I_k = \int_A^B x_{ki} {\rm d}i$$ (D.2)

with an error estimate $\sigma_{I_k}$ given by:

$$\sigma_{I_k}^2 = \int_A^B \int_A^B \sigma_{ij}^2 {\rm d}j {\rm d}i.$$ (D.3)

Only if all points within (A,B) behave statistically independently this reduces to the commonly used variance-deduced error estimate, because then:

$$\sigma_{ij} = \sigma_{ii} \delta_{ij}$$ (D.4)

with $\delta_{ij}$ the Kronecker delta function. In the opposite extreme when all points within (A,B) behave in phase:

$$\sigma_{ij} = \sigma_{ii}$$ (D.5)

and consequently no increase in signal-to-noise can be obtained by integrating along the spectrum. The integrated covariances $\int_A^B \sigma_{ij}^2 {\rm d}j$ are calculated in spectral regions without intrinsic spectral variability (Table D.1) and tabulated in flux-wavelength space. Errors can then be assigned according to flux and wavelength. This assumes that the integrated covariances are a smooth function of flux and wavelength, which was proven to be true for IUE spectra (Howarth & Smith 1995).

 

 

Table D.1: Intervals (A,B), in Å, used for error estimation.

Cyg X-1	LMC X-4	SMC X-1	Vela X-1	4U1700-37
1250,1370	1170,1200	1250,1380	1153,1200	1183,1225
1420,1510	1280,1380	1415,1530	1253,1280	1420,1517
1565,1926	1430,1530	1560,1926	1315,1380	1730,1947
	1565,1926		1415,1530
			1565,1840

Figure D.1: Covariance (top) and cumulative covariance (bottom) for Cyg X-1 (left) and Vela X-1 (right) as a function of distance along the spectrum with respect to the spectral point i. The drawn, dashed, dotted and dash-dotted lines represent the averages for the entire spectrum and the three spectral regions in which the continuum calibration factors were determined, respectively, with subsequently larger mean wavelength. The vertical long-dashed line indicates the choice of the integration boundary.

Figure D.2: Variances (top) and integrated covariances (bottom) in the spectra of Cyg X-1 (left) and Vela X-1 (right), as a function of wavelength of the spectral points i.

The covariances $\sigma_{ij}^2$ decrease with increasing distance |j-i| to the point i. If the integral of the covariances for the point i converges sufficiently fast, then the integration interval (A,B) in Eq. (D3) may be replaced by a smaller interval such that the integral of the covariances just reaches convergence. This interval is determined from a covariance profile of $\sigma_{ij}^2$ versus j-i, which represents the spectral shape of the instrumental profile. As an example the covariance profile and the cumulative covariance are shown for Cyg X-1 and Vela X-1 (Fig. D.1). The dashed, dotted and dash-dotted lines represent the mean curves within the three regions in which the continuum calibration factors were determined (from shortest to longest wavelength, respectively), whilst the solid line represents the entire spectral range - i.e. including wildly variable spectral features. Both the covariance and the cumulative covariance at zero distance from the point ireduce to the variance - i.e. the covariance of point i with itself. At larger distances the covariance diminishes to a small oscillation around zero, whilst the initially rapidly growing cumulative covariance approximates a constant level.

For Cyg X-1 the solid line is too high to be captured within the frame of the plots, indicating that some parts of the spectrum are strongly variable. The width of the covariance profile corresponds to the specified spectral resolving power of about 250. The cumulative covariance indicates that a reasonable choice for the integration interval is $\pm6$ Å. The sharp peak of the covariance profile of Vela X-1 indicates a spectral resolution in accordance with the specified spectral resolving power of about 10⁴. Its broad wings, however, prevent the cumulative covariance from converging within $\sim$16 spectral points ($\equiv$1.6 Å) - several times the specified spectral resolution.

Spectra of the variances and integrated covariances are shown for Cyg X-1 and Vela X-1 (Fig. D.2) as an example of the detection of intrinsic spectral variability. In the low-resolution spectra of Cyg X-1 the spectral features (notably the Si  IV and C  IV resonance lines near 1400 and 1550 Å, respectively) and their variability are unresolved. The calculation of covariances does not improve the detection sensitivity much. In the high-resolution spectra of Vela X-1, however, the integrated covariance spectrum is much more sensitive than the variance spectrum in detecting intrinsic variability. Events that are limited to only one or very few adjacent pixels (like cosmics or dead pixels) cause a "forest'' of sharp peaks in the variance spectrum, nearly completely obscuring the resolved but still narrow peaks of the intrinsic variability - the "trees'' of interest. By calculating the integrated covariance spectrum the forest is suppressed, and at the same time the features indicating intrinsic variability are enhanced. The variable spectral features can be identified easily.