Subscriber Authentication Point
Free Access
 Issue A&A Volume 506, Number 1, October IV 2009 The CoRoT space mission: early results 1 - 5 Stellar structure and evolution https://doi.org/10.1051/0004-6361/200810990 22 June 2009

The CoRoT space mission: early results

## On posterior probability and significance level: application to the power spectrum of HD 49 933 observed by CoRoT

T. Appourchaux1 - R. Samadi2 - M.-A. Dupret2,3

1 - Institut d'Astrophysique Spatiale, UMR8617, Université Paris XI, Bâtiment 121, 91405 Orsay Cedex, France
2 - Observatoire de Paris, LESIA, UMR8109, 92195 Meudon Cedex, France
3 - Institut d'Astrophysique et de Géophysique de l'Université de Liège, Allée du 6 Août 17, 4000 Liège, Belgium

Received 19 September 2008 / Accepted 21 May 2009

Abstract
Context. The CoRoT mission provides asteroseismic data of very high quality allowing one to adopt new statistical approaches for mode detection in power spectra, especially with respect to testing the null hypothesis (H0, which assumes that what is observed is pure noise).
Aims. We emphasize that the significance level when rejecting the null hypothesis can lead to the incorrect conclusion that the H0 hypothesis is unlikely to occur at that significance level. We demonstrate that the significance level is unrelated to the posterior probability of H0, given the observed data set, and that this posterior probability is very much higher than implied by the significance level.
Methods. We use Bayes theorem to derive the posterior probability of that H0 is true assuming an alternative hypothesis H1 that a mode is present, taking some prior for the mode height, mode amplitude and linewidth.
Results. We compute the posterior probability of H0 for the p modes detected on HD 49 933 by CoRoT.
Conclusions. We conclude that the posterior probability of H0 provide a much more conservative quantification of the mode detection than the significance level. This framework can be applied to any similar stellar power spectra obtained to complete asteroseismology.

Key words: methods: analytical - methods: statistical - Sun: helioseismology

## 1 Introduction

In the field of helioseismology, the null hypothesis H0 was used by Appourchaux et al. (2000) to infer upper limits to the amplitude of g modes. The H0 hypothesis assumes that what is observed is pure noise. This hypothesis was applied to data of classical variable stars to detect peaks in their power spectra (Scargle 1982). In all of these papers, the authors assumed H0 and defined an upper limit corresponding to a threshold level of, e.g., 10%. Here we argue, that this threshold was arbitrarily chosen a priori. When accepting or rejecting the H0 hypothesis, there is no discussion whether borderline case should be rejected or accepted. This abrupt truncation between good and bad leads to a decision that could have been different if the threshold had been different.

In the following sections, we first lay the foundations for understanding the meaning of the H0 hypothesis, and then explain what is commonly misunderstood about the H0 hypothesis. We then derive, for specific cases encountered in helio- and asteroseismology, the posterior probability of H0. We show how one can apply the formalism for real data, such as that gathered by the CoRoT mission for HD 49 933 and then present our conclusions.

## 2 Significance level and the H0 hypothesis

Fisher (1925) devised the well known Fisher test for testing the null hypothesis (H0). In this test, a threshold of 5% is commonly used and the p-value quoted is the value of the test if it is less than this threshold. For instance, a result of 4.9% would result in rejecting the H0 hypothesis, while 5.1% would result in accepting the H0 hypothesis. When the H0 hypothesis is rejected, the reported p-value is used as a significance level for the validating or not accepting the H0 hypothesis. In the medical field, the so-called borderline cases led to findings related to effectiveness of medicine that were, sometimes, not proven by subsequent studies. The controversy about the use of p-values that occurred in the medical world is directly related to the abrupt and arbitrary cut-off of the threshold applied (be 5% or 10%). Although the relevance of what has been found in the medical field could seem remote to most astrophysicists, it is extremely relevant to understand that the improper use of p-values is the same as the improper use of the so-called significance level.

Reporting a small number for the significance level should not be used to claim the proper rejection of the H0 hypothesis. The mistake is to ascribe a significance level to a measurement carried out only once, not repeated and spanning just a very small volume of the space of the parameters. When making an observation of a random variable x, one wants to check the probability that what is observed could be due to noise. For that purpose, a test statistic is derived called T(x). If one observes a value of x=X that would not reject the H0, then one can compute, p, the significance level or p-value, defined to be:

 (1)

where P0 is the probability that when H0 is true. The test set by Eq. (1) is to confirm that the statistical test T(x) has been compared with the value given by T(X). When computing the statistical test T(x), one does not span the space of X: one has x=X which is the so-called point null hypothesis. In other terms, what is the probability that one has exactly that value of x? This is a completely different question from knowing how true H0 is, i.e., obtaining . Berger & Sellke (1987) provided a way of deriving with respect to and , where H1 is the alternative hypothesis (i.e., there is a signal). Using Bayes theorem, they obtained the so-called posterior probability of H0 given the observed data x:

 (2)

where and are the probabilities related to the H0 and H1 hypothesis, respectively. We note that we would like to determine , the probability of H0 being true given the data we have, not the probability of the data x given H0, i.e., . To minimize the impact of the probabilities assigned to both hypotheses, we assume that these are equally probable (p0=0.5):

 (3)

Based under these assumptions, Sellke et al. (2001) found that the probability that H0 is true given some observed data x of a random variable X has a lower bound

 (4)

An immediate consequence is that for a significance level of 1%, the odds against H0 are at least 10 to 1, and for 10%, the odds against H0 are at least 2.6 to 1. In both cases, the likelihood of incorrectly rejecting H0 is much higher than inferred by the p-value, by at most a factor of 10 and 4, respectively.

Sellke et al. (2001) were able to set a lower value to for almost an arbitrary alternative H1 hypothesis. We show that there is indeed a lower bound when one wants to detect peaks in power spectra. Hereafter, we give several examples of how one can derive in practice the posterior probability of H0.

## 3 Posterior probability for peak detection

Here we provide analytical examples of how one can obtain the posterior probability of H0 for some cases encountered in astero- and helio-seismology.

### 3.1 Long-lived modes

#### 3.1.1 Mode height known a priori

We search a peak restricted to a single frequency bin in a power spectrum which has a  with 2 d.o.f. statistics, for which a bin has reached a value x. We wish to verify whether this is caused by either a true sine wave or due to noise. We have for the H0 hypothesis,

 (5)

The noise is assumed to be 1. For the alternative hypothesis H1, we assume that there is a signal of a long-lived mode, i.e., restricted to one bin, for which the mode height H is known and the mode is stochastically excited (like a stellar p mode). We then have:

 (6)

Equation (3) is then rewritten for our problem as

 (7)

Since the significance , we finally have that

 (8)

It can be shown that the minimum of is reached for for the value:

 (9)

In this case, the lower bound defined by Eq. (4) is reached.

#### 3.1.2 Mode height unknown

Most of the time one does not know of course the height of the mode to be detected. We can assume a prior for the mode height which can be e.g., a uniform distribution or a gaussian function. For example, if we assume that the mode height is uniformly distributed over some range [0, ], is rewritten as

 (10)

Then Eq. (3) can be rewritten as

 (11)

It can be shown that a minimum is reached at a value of by solving

 (12)

The minimum is then given by

 (13)

From Eqs. (8) and (9), we can deduce that

 (14)

This means that when there is more uncertainty about the possible height of the mode, one is less likely to reject the hypothesis.

### 3.2 Short-lived modes

Appourchaux (2004) described how one can detect a mode having a lifetime shorter than the observation time. He suggested smoothing the power spectrum to increase the signal-to-noise ratio and provided an analytical expression for H0 related to the summation over n bins of a  statistic with constant mean, and for H1 related to the summation over n bins of a  statistic with different means due to the presence of the mode profile. Therefore, when the spectrum is summed over n bins, we can derive for the H0 hypothesis

 (15)

where the mean of the power spectrum is 1, while n is the mean of the smoothed power spectrum (for simplicity, we assumed that S=1), and is the Gamma function. The significance level x is then given by solving

 (16)

Assuming that the mode has a known amplitude A and a known linewidth , we can write using Eq. (8) of Appourchaux (2004) the approximation

 (17)

where and , defined in Appourchaux (2004) are obtained by integrating symmetrically about the central frequency of the mode, thereby ensuring that the signal is maximum; and  are functions of the mode height and . If we do not know the mode amplitude and its linewidth, we can specify a prior for the amplitude and the linewidth, which, for instance, can be done using a model of mode excitation. Here we assume that the mode amplitude A and the linewidth  are independent (this would not be the case of H and ). If we use uniform priors, we then have

 (18)

This equation would be similar if we were to have a uniform prior on the mode height, where A would then be replaced by H. Substituting Eqs. (16) and (17) into Eq. (3), we can then obtain the posterior probability when the mode amplitude and linewidth are known. If we replace Eqs. (17) by (18), we can then obtain the probability  when the mode amplitude and linewidth are not known.
 Figure 1: Posterior probability as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) for a significance level of 10%. For the known mode height, the minimum is reached for an height of 1.3 with a value of 38%. For the unknown mode height, the minimum is reached at a value of 40%. Posterior probability as a function of the known height (dashed line), or as a function of the maximum mode height of the uniform prior (continuous line) for a significance level of 1%. For the known mode height, the minimum is reached for an height of 3.6 with a value of 11%. For the unknown mode height, the minimum is reached at a value of 12.7%. Open with DEXTER

### 3.3 Discussion

Figure 1 shows the posterior probability for long-lived modes of a known mode height (Eq. (8)) and an unknown mode height (Eq. (11)) for two different significance levels. Figure 2 shows the posterior probability for short-lived modes of a known mode height and linewidth (Eqs. (15) and (17)) and for an unknown mode height and linewidth (Eqs. (15) and (18), with H replacing A) for a single significance level; even a significance level of 1% does not provide a more robust rejection of the H0 hypothesis. These posterior probabilities have a lower bound which means that even a very low significance level is no guarantee for positive detection!

It is also counterintuitive that the posterior probability increases when the mode height (known or unknown) increases. We recall that the significance level p corresponds to the level x at which the peak has been observed (i.e., it is for long-lived modes). If we assume a priori that the mode has a high mode height, then the observation at a low significance level indicates that our assumption about the high mode height is incorrect, and that the data dismisses the a priori assumptions made about the mode height. In other words, it is more probable that the H0 hypothesis is true.

In the absence of an alternative hypothesis, it is advisable to define a low significance value that will probably reject the H0 hypothesis. Figure 3 shows the lower bound set by Eq. (4) compared to the minimum found using Eq. (11) for the uniform prior on the mode height.

 Figure 2: Posterior probability for a short-lived mode after smoothing the power spectra over 10 bins corresponding to a window twice as large as the mode linewidth. The posterior probability is given as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) and a uniform prior for the linewidth, for a significance level of 10%. Open with DEXTER

 Figure 3: Lower bound to the posterior probability as function of the significance level for the known mode height (dashed line) and for the uniform prior (solid line). Open with DEXTER

## 4 Application to the CoRoT data: HD 49 933

The computation of the posterior probability was applied to CoRoT data for illustrative purposes. The data used are those of the first initial run performed on HD 49 933 (Appourchaux et al. 2008). The objective was to provide an objective way of detecting oscillation modes in HD 49 933 that could be applied to any other star.

The methodology used for deriving the posterior probability was as follows:

• we computed the power spectrum from the detrended time series, as in Appourchaux et al. (2008);
• we smoothed the spectrum over n bins using a boxcar;
• we selected 30 50-Hz wide windows starting at 1200 Hz (the 50-Hz window corresponds roughly to half the large frequency separation);
• for each window, we computed the median in the window of the smoothed spectrum which provides an estimate of the mean noise level if the modes are not present;
• the smoothed spectrum was normalized in each window by dividing by the median and multiplying by the number of smoothing bins n which provides values commensurate with these of Eq. (15);
• we applied the H0 hypothesis for a detection probability that a signal caused by to noise of 10% over all the 30 windows, taking into account the fact that in each window the number of independent bins is 50 ( is the frequency resolution of the original power spectrum). The detection probability is then 0.1 per independent bin;
• we then solved Eq. (16) for given the detection level given above;
• in each window, we then selected the bins that are greater than  , i.e., we accepted or rejected the H0 hypothesis;
• after the selection, we retained the greatest value  found in the window corresponding to the central frequency of the mode (See Eq. (17));
• we then computed the posterior probability of H0 given by Eq. (3) using Eqs. (15) and (18) assuming some prior on the mode height and linewidth as described below;
• for comparison, we also computed the significance level as given by Eq. (16) from the value of  .
The theoretical amplitudes for HD 49 933 are derived from Samadi et al. (2009a) using an adiabatic treatment of radiative transfer, and the excitation rate as calculated in Samadi et al. (2009b). The theoretical linewidths were computed with the non-adiabatic pulsation code MAD. This code includes a time-dependent convection treatment described in Grigahcène et al. (2005): it takes into account the role played by the variations in the convective flux, the turbulent pressure, and the dissipation rate of turbulent kinetic energy. This treatment is non-local, with three free parameters a, b, and c corresponding to the non-locality of the convective flux, the turbulent pressure, and the entropy gradient. We assume the values a=10, b=3, and c=3.5 obtained by fitting the convective flux and turbulent pressure of 3D hydrodynamic simulations in the upper overshooting region of the Sun (Dupret et al. 2006). According to Grigahcène et al. (2005), we introduced a free complex parameter  in the perturbation of the energy closure equation. We used here the value , which provides a good agreement between the theoretical and observed linewidths and phase lags in the range of solar pressure modes. A 1D stellar model obtained with the code CESAM was used for our non-adiabatic computations. It has a solar metallicity and reproduces the effective temperature and gravity of HD 49 933 (Samadi et al. 2009a). The amplitudes and linewidths are shown on Fig. 4. We linearly interpolated both curves to provide a continuous prior with frequency.
 Figure 4: ( Left) Theoretical mode linewidth of HD 49 933 as a function of frequency. ( Right) Theoretical mode amplitude of HD 49 933 as a function of frequency. Open with DEXTER

 Figure 5: ( Top) Comparison of the posterior probability (solid line) with the significance level (dashed line) as a function of frequency for a spectrum smoothed over 5 bins (Hz). ( Bottom) Posterior probability as a function of frequency for a spectrum smoothed over 5 bins (solid line) and over 50 bins (dashed line), of sizes Hz and Hz, respectively. Open with DEXTER

The uniform priors for amplitude and linewidth are derived from the theory by taking into account an uncertainty factor in the theoretical model. For amplitude, we assumed that the maximum is larger than given in Fig. 4 (twice in energy); for linewidth, we assumed that the maximum is twice as great as that given in Fig. 4. We note that a larger prior increases the posterior probability, as shown on Figs. 1 and 2. We assumed that the noise floor in HD 49 933 is given by the photon noise, which is about 0.15 ppmHz (Appourchaux et al. 2008).

Figure 5 shows the results of the procedure described above. It is clear that the posterior probability is higher than the significance level, i.e., the posterior probability provides a moreconservative number (H0 more likely). The smoothing procedure also shows two effects that were predicted by Appourchaux (2004): first, short lived modes are easier to detect when the spectrum is smoothed, second, long lived modes are more difficult to detect when the spectrum is smoothed. The first effect manifests itself in the larger number of detected modes at higher frequency and by the decrease of the prior probability (i.e., the signal is more likely). The second effect is seen at low frequency where a couple of modes have their prior probability increased to non-negligible value after smoothing (i.e., the signal is less likely). When we compare with modes reported by Appourchaux et al. (2008), we find that more than 85% of the l=0-2 mode pairs and l=1 modes are recovered. An additional mode at 2579 Hz is detected that could be an l=1 mode according to the identification of Appourchaux et al. (2008).

## 5 Conclusion

The significance level refers to the significance of the data given the hypothesis, while we are interested in the posterior probability of the null hypothesis given the data. Here we have shown that for a significance level of 10%, the posterior probability of the null hypothesis is at least 38% when there is no alternative hypothesis. We have illustrated how one can in practice calculate and compute the posterior probability for the null hypothesis. This has been applied to several theoretical examples and to the CoRoT data. For the first time, we have shown how one can assess the detectability of short-lived p modes in a power spectrum. The methodology can be applied to any stellar power spectrum for which theoretical expectations are available.

Acknowledgements
We are grateful to John Leibacher for checking the English and for correcting several typos. We are thankful to the referee for making the paper clearer.

## Footnotes

... CoRoT
The CoRoT space mission, launched on 2006 December 27, was developed and is operated by the CNES, with participation of the Science Programs of ESA, ESA's RSSD, Austria, Belgium, Brazil, Germany and Spain.
... bound
The same lower bound as Eq. (9) for a known mode height.

## All Figures

 Figure 1: Posterior probability as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) for a significance level of 10%. For the known mode height, the minimum is reached for an height of 1.3 with a value of 38%. For the unknown mode height, the minimum is reached at a value of 40%. Posterior probability as a function of the known height (dashed line), or as a function of the maximum mode height of the uniform prior (continuous line) for a significance level of 1%. For the known mode height, the minimum is reached for an height of 3.6 with a value of 11%. For the unknown mode height, the minimum is reached at a value of 12.7%. Open with DEXTER In the text

 Figure 2: Posterior probability for a short-lived mode after smoothing the power spectra over 10 bins corresponding to a window twice as large as the mode linewidth. The posterior probability is given as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) and a uniform prior for the linewidth, for a significance level of 10%. Open with DEXTER In the text

 Figure 3: Lower bound to the posterior probability as function of the significance level for the known mode height (dashed line) and for the uniform prior (solid line). Open with DEXTER In the text

 Figure 4: ( Left) Theoretical mode linewidth of HD 49 933 as a function of frequency. ( Right) Theoretical mode amplitude of HD 49 933 as a function of frequency. Open with DEXTER In the text

 Figure 5: ( Top) Comparison of the posterior probability (solid line) with the significance level (dashed line) as a function of frequency for a spectrum smoothed over 5 bins (Hz). ( Bottom) Posterior probability as a function of frequency for a spectrum smoothed over 5 bins (solid line) and over 50 bins (dashed line), of sizes Hz and Hz, respectively. Open with DEXTER In the text