Issue |
A&A
Volume 506, Number 1, October IV 2009
The CoRoT space mission: early results
|
|
---|---|---|
Page(s) | 1 - 5 | |
Section | Stellar structure and evolution | |
DOI | https://doi.org/10.1051/0004-6361/200810990 | |
Published online | 22 June 2009 |
The CoRoT space mission: early results
On posterior probability and significance level: application to the power spectrum of HD 49 933 observed by CoRoT![[*]](/icons/foot_motif.png)
T. Appourchaux1 - R. Samadi2 - M.-A. Dupret2,3
1 - Institut d'Astrophysique Spatiale, UMR8617, Université Paris XI, Bâtiment 121, 91405 Orsay Cedex, France
2 -
Observatoire de Paris, LESIA, UMR8109, 92195 Meudon Cedex, France
3 -
Institut d'Astrophysique et de Géophysique de l'Université de Liège, Allée du 6 Août 17, 4000 Liège, Belgium
Received 19 September 2008 / Accepted 21 May 2009
Abstract
Context. The CoRoT mission provides asteroseismic data of very high quality allowing one to adopt new statistical approaches for mode detection in power spectra, especially with respect to testing the null hypothesis (H0, which assumes that what is observed is pure noise).
Aims. We emphasize that the significance level when rejecting the null hypothesis can lead to the incorrect conclusion that the H0 hypothesis is unlikely to occur at that significance level. We demonstrate that the significance level is unrelated to the posterior probability of H0, given the observed data set, and that this posterior probability is very much higher than implied by the significance level.
Methods. We use Bayes theorem to derive the posterior probability of that H0 is true assuming an alternative hypothesis H1 that a mode is present, taking some prior for the mode height, mode amplitude and linewidth.
Results. We compute the posterior probability of H0 for the p modes detected on HD 49 933 by CoRoT.
Conclusions. We conclude that the posterior probability of H0 provide a much more conservative quantification of the mode detection than the significance level. This framework can be applied to any similar stellar power spectra obtained to complete asteroseismology.
Key words: methods: analytical - methods: statistical - Sun: helioseismology
1 Introduction
In the field of helioseismology, the null hypothesis H0 was used by Appourchaux et al. (2000) to infer upper limits to the amplitude of g modes. The H0 hypothesis assumes that what is observed is pure noise. This hypothesis was applied to data of classical variable stars to detect peaks in their power spectra (Scargle 1982). In all of these papers, the authors assumed H0 and defined an upper limit corresponding to a threshold level of, e.g., 10%. Here we argue, that this threshold was arbitrarily chosen a priori. When accepting or rejecting the H0 hypothesis, there is no discussion whether borderline case should be rejected or accepted. This abrupt truncation between good and bad leads to a decision that could have been different if the threshold had been different.In the following sections, we first lay the foundations for understanding the meaning of the H0 hypothesis, and then explain what is commonly misunderstood about the H0 hypothesis. We then derive, for specific cases encountered in helio- and asteroseismology, the posterior probability of H0. We show how one can apply the formalism for real data, such as that gathered by the CoRoT mission for HD 49 933 and then present our conclusions.
2 Significance level and the H0 hypothesis
Fisher (1925) devised the well known Fisher test for testing the null hypothesis (H0). In this test, a threshold of 5% is commonly used and the p-value quoted is the value of the test if it is less than this threshold. For instance, a result of 4.9% would result in rejecting the H0 hypothesis, while 5.1% would result in accepting the H0 hypothesis. When the H0 hypothesis is rejected, the reported p-value is used as a significance level for the validating or not accepting the H0 hypothesis. In the medical field, the so-called borderline cases led to findings related to effectiveness of medicine that were, sometimes, not proven by subsequent studies. The controversy about the use of p-values that occurred in the medical world is directly related to the abrupt and arbitrary cut-off of the threshold applied (be 5% or 10%). Although the relevance of what has been found in the medical field could seem remote to most astrophysicists, it is extremely relevant to understand that the improper use of p-values is the same as the improper use of the so-called significance level.
Reporting a small number for the significance level should not be used to claim the proper rejection of the H0 hypothesis. The mistake is to ascribe a significance level to a measurement carried out only once, not repeated and spanning just a very small volume of the space of the parameters. When making an observation of a random variable x, one wants to check the probability that what is observed could be due to noise. For that purpose, a test statistic is derived called T(x). If one observes a value of x=X that would not reject the H0, then one can compute, p, the significance level or p-value, defined to be:
![]() |
(1) |
where P0 is the probability that





![]() |
(2) |
where




![]() |
(3) |
Based under these assumptions, Sellke et al. (2001) found that the probability

![]() |
(4) |
An immediate consequence is that for a significance level of 1%, the odds against H0 are at least 10 to 1, and for 10%, the odds against H0 are at least 2.6 to 1. In both cases, the likelihood of incorrectly rejecting H0 is much higher than inferred by the p-value, by at most a factor of 10 and 4, respectively.
Sellke et al. (2001) were able to set a lower value to
for almost an arbitrary alternative H1 hypothesis. We show that there is indeed a lower bound when one wants to detect peaks in power spectra. Hereafter, we give several examples of how one can derive in practice the posterior probability of H0.
3 Posterior probability for peak detection
Here we provide analytical examples of how one can obtain the posterior probability of H0 for some cases encountered in astero- and helio-seismology.
3.1 Long-lived modes
3.1.1 Mode height known a priori
We search a peak restricted to a single frequency bin in a power spectrum which has a
with 2 d.o.f. statistics, for which a bin has reached a value x. We wish to verify whether this is caused by either a true sine wave or due to noise. We have for the H0 hypothesis,
![]() |
(5) |
The noise is assumed to be 1. For the alternative hypothesis H1, we assume that there is a signal of a long-lived mode, i.e., restricted to one bin, for which the mode height H is known and the mode is stochastically excited (like a stellar p mode). We then have:
![]() |
(6) |
Equation (3) is then rewritten for our problem as
![]() |
(7) |
Since the significance

![]() |
(8) |
It can be shown that the minimum of


![]() |
(9) |
In this case, the lower bound defined by Eq. (4) is reached.
3.1.2 Mode height unknown
Most of the time one does not know of course the height of the mode to be detected. We can assume a prior for the mode height which can be e.g., a uniform distribution or a gaussian function. For example, if we assume that the mode height is uniformly distributed over some range [0,

![]() |
(10) |
Then Eq. (3) can be rewritten as
![]() |
(11) |
It can be shown that a minimum is reached at a value of

![]() |
(12) |
The minimum is then given by
![]() |
(13) |
From Eqs. (8) and (9), we can deduce that
![]() |
(14) |
This means that when there is more uncertainty about the possible height of the mode, one is less likely to reject the

3.2 Short-lived modes
Appourchaux (2004) described how one can detect a mode having a lifetime shorter than the observation time. He suggested smoothing the power spectrum to increase the signal-to-noise ratio and provided an analytical expression for H0 related to the summation over n bins of a

![]() |
(15) |
where the mean of the power spectrum is 1, while n is the mean of the smoothed power spectrum (for simplicity, we assumed that S=1), and

Assuming that the mode has a known amplitude A and a known linewidth

![]() |
(17) |
where








![]() |
(18) |
This equation would be similar if we were to have a uniform prior on the mode height, where A would then be replaced by H. Substituting Eqs. (16) and (17) into Eq. (3), we can then obtain the posterior probability


![]() |
Figure 1:
Posterior probability as a function of the known mode height (dashed line), or as a function of the mode height range ( |
Open with DEXTER |
3.3 Discussion
Figure 1 shows the posterior probability for long-lived modes of a known mode height (Eq. (8)) and an unknown mode height (Eq. (11)) for two different significance levels. Figure 2 shows the posterior probability for short-lived modes of a known mode height and linewidth (Eqs. (15) and (17)) and for an unknown mode height and linewidth (Eqs. (15) and (18), with H replacing A) for a single significance level; even a significance level of 1% does not provide a more robust rejection of the H0 hypothesis. These posterior probabilities have a lower bound which means that even a very low significance level is no guarantee for positive detection!
It is also counterintuitive that the posterior probability increases when the mode height (known or unknown) increases. We recall that the significance level p corresponds to the level x at which the peak has been observed (i.e., it is
for long-lived modes). If we assume a priori that the mode has a high mode height, then the observation at a low significance level indicates that our assumption about the high mode height is incorrect, and that the data dismisses the a priori assumptions made about the mode height. In other words, it is more probable that the H0 hypothesis is true.
In the absence of an alternative hypothesis, it is advisable to define a low significance value that will probably reject the H0 hypothesis. Figure 3 shows the lower bound set by Eq. (4) compared to the minimum found using Eq. (11) for the uniform prior on the mode height.
![]() |
Figure 2:
Posterior probability for a short-lived mode after smoothing the power spectra over 10 bins corresponding to a window twice as large as the mode linewidth. The posterior probability is given as a function of the known mode height (dashed line), or as a function of the mode height range ( |
Open with DEXTER |
![]() |
Figure 3: Lower bound to the posterior probability as function of the significance level for the known mode height (dashed line) and for the uniform prior (solid line). |
Open with DEXTER |
4 Application to the CoRoT data: HD 49 933
The computation of the posterior probability was applied to CoRoT data for illustrative purposes. The data used are those of the first initial run performed on HD 49 933 (Appourchaux et al. 2008). The objective was to provide an objective way of detecting oscillation modes in HD 49 933 that could be applied to any other star.The methodology used for deriving the posterior probability was as follows:
- we computed the power spectrum from the detrended time series, as in Appourchaux et al. (2008);
- we smoothed the spectrum over n bins using a boxcar;
- we selected 30 50-
Hz wide windows starting at 1200
Hz (the 50-
Hz window corresponds roughly to half the large frequency separation);
- for each window, we computed the median in the window of the smoothed spectrum which provides an estimate of the mean noise level if the modes are not present;
- the smoothed spectrum was normalized in each window by dividing by the median and multiplying by the number of smoothing bins n which provides values commensurate with these of Eq. (15);
- we applied the H0 hypothesis for a detection probability that a signal caused by to noise of 10% over all the 30 windows, taking into account the fact that in each window the number of independent bins is 50
(
is the frequency resolution of the original power spectrum). The detection probability is then 0.1
per independent bin;
- we then solved Eq. (16) for
given the detection level given above;
- in each window, we then selected the bins that are greater than
, i.e., we accepted or rejected the H0 hypothesis;
- after the selection, we retained the greatest value
found in the window corresponding to the central frequency of the mode (See Eq. (17));
- we then computed the posterior probability of H0 given by Eq. (3) using Eqs. (15) and (18) assuming some prior on the mode height and linewidth as described below;
- for comparison, we also computed the significance level as given by Eq. (16) from the value of
.


![]() |
Figure 4: ( Left) Theoretical mode linewidth of HD 49 933 as a function of frequency. ( Right) Theoretical mode amplitude of HD 49 933 as a function of frequency. |
Open with DEXTER |
![]() |
Figure 5:
( Top) Comparison of the posterior probability (solid line) with the significance level (dashed line) as a function of frequency for a spectrum smoothed over 5 bins ( |
Open with DEXTER |
The uniform priors for amplitude and linewidth are derived from the theory by taking into account an uncertainty factor in the theoretical model. For amplitude, we assumed that the maximum is
larger than given in Fig. 4 (twice in energy); for linewidth, we assumed that the maximum is twice as great as that given in Fig. 4. We note that a larger prior increases the posterior probability, as shown on
Figs. 1 and 2. We assumed that the noise floor in
HD 49 933 is given by the photon noise, which is about 0.15 ppm
Hz (Appourchaux et al. 2008).
Figure 5 shows the results of the procedure described above. It is clear that the posterior probability is higher than the significance level, i.e., the posterior probability provides a moreconservative number (H0 more likely). The smoothing procedure also shows two effects that were predicted by Appourchaux (2004): first, short lived modes are easier to detect when the spectrum is smoothed, second, long lived modes are more difficult to detect when the spectrum is smoothed. The first effect manifests itself in the larger number of detected modes at higher frequency and by the decrease of the prior probability (i.e., the signal is more likely). The second effect is seen at low frequency where a couple of modes have their prior probability increased to non-negligible value after smoothing (i.e., the signal is less likely). When we compare with modes reported by Appourchaux et al. (2008), we find that more than 85% of the l=0-2 mode pairs and l=1 modes are recovered. An additional mode at 2579 Hz is detected that could be an l=1 mode according to the identification of Appourchaux et al. (2008).
5 Conclusion
The significance level refers to the significance of the data given the hypothesis, while we are interested in the posterior probability of the null hypothesis given the data. Here we have shown that for a significance level of 10%, the posterior probability of the null hypothesis is at least 38% when there is no alternative hypothesis. We have illustrated how one can in practice calculate and compute the posterior probability for the null hypothesis. This has been applied to several theoretical examples and to the CoRoT data. For the first time, we have shown how one can assess the detectability of short-lived p modes in a power spectrum. The methodology can be applied to any stellar power spectrum for which theoretical expectations are available.
Acknowledgements
We are grateful to John Leibacher for checking the English and for correcting several typos. We are thankful to the referee for making the paper clearer.
References
- Appourchaux, T. 2004, A&A, 428, 1039 [NASA ADS] [CrossRef] [EDP Sciences] (In the text)
- Appourchaux, T., Fröhlich, C., Andersen, B., et al. 2000, ApJ, 538, 401 [NASA ADS] [CrossRef] (In the text)
- Appourchaux, T., Michel, E., Auvergne, M., et al. 2008, A&A, 488, 705 [CrossRef] [EDP Sciences] (In the text)
- Berger, J., & Sellke, T. 1987, J. Am. Stat. Assoc., 82(397), 112 (In the text)
- Dupret, M.-A., Samadi, R., Grigahcene, A., Goupil, M.-J., & Gabriel, M. 2006, Commun. Asteroseismol., 147, 85 [CrossRef] (In the text)
- Fisher, R. A. 1925, Statistical Methods for Research Workers (Edinburgh, Scotland: Oliver and Boyd), 299 (In the text)
- Grigahcène, A., Dupret, M.-A., Gabriel, M., Garrido, R., & Scuflaire, R. 2005, A&A, 434, 1055 [NASA ADS] [CrossRef] [EDP Sciences] (In the text)
- Samadi, R., Ludwig, H.-G., Belkacem, K., et al. 2009a, A&A, submitted (In the text)
- Samadi, R., Ludwig, H.-G., Belkacem, K., Goupil, M., & Dupret, M.-A. 2009b, A&A, submitted (In the text)
- Scargle, J. D. 1982, ApJ, 263, 835 [NASA ADS] [CrossRef] (In the text)
- Sellke, T., Bayarri, M. J., & Berger, J. 2001, The American Statistician, 55, 62 [CrossRef] (In the text)
Footnotes
- ... CoRoT
- The CoRoT space mission, launched on 2006 December 27, was developed and is operated by the CNES, with participation of the Science Programs of ESA, ESA's RSSD, Austria, Belgium, Brazil, Germany and Spain.
- ... bound
- The same lower bound as Eq. (9) for a known mode height.
All Figures
![]() |
Figure 1:
Posterior probability as a function of the known mode height (dashed line), or as a function of the mode height range ( |
Open with DEXTER | |
In the text |
![]() |
Figure 2:
Posterior probability for a short-lived mode after smoothing the power spectra over 10 bins corresponding to a window twice as large as the mode linewidth. The posterior probability is given as a function of the known mode height (dashed line), or as a function of the mode height range ( |
Open with DEXTER | |
In the text |
![]() |
Figure 3: Lower bound to the posterior probability as function of the significance level for the known mode height (dashed line) and for the uniform prior (solid line). |
Open with DEXTER | |
In the text |
![]() |
Figure 4: ( Left) Theoretical mode linewidth of HD 49 933 as a function of frequency. ( Right) Theoretical mode amplitude of HD 49 933 as a function of frequency. |
Open with DEXTER | |
In the text |
![]() |
Figure 5:
( Top) Comparison of the posterior probability (solid line) with the significance level (dashed line) as a function of frequency for a spectrum smoothed over 5 bins ( |
Open with DEXTER | |
In the text |
Copyright ESO 2009
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.