Issue 
A&A
Volume 506, Number 1, October IV 2009
The CoRoT space mission: early results



Page(s)  1  5  
Section  Stellar structure and evolution  
DOI  https://doi.org/10.1051/00046361/200810990  
Published online  22 June 2009 
The CoRoT space mission: early results
On posterior probability and significance level: application to the power spectrum of HD 49 933 observed by CoRoT^{}
T. Appourchaux^{1}  R. Samadi^{2}  M.A. Dupret^{2,3}
1  Institut d'Astrophysique Spatiale, UMR8617, Université Paris XI, Bâtiment 121, 91405 Orsay Cedex, France
2 
Observatoire de Paris, LESIA, UMR8109, 92195 Meudon Cedex, France
3 
Institut d'Astrophysique et de Géophysique de l'Université de Liège, Allée du 6 Août 17, 4000 Liège, Belgium
Received 19 September 2008 / Accepted 21 May 2009
Abstract
Context. The CoRoT mission provides asteroseismic data of very high quality allowing one to adopt new statistical approaches for mode detection in power spectra, especially with respect to testing the null hypothesis (H_{0}, which assumes that what is observed is pure noise).
Aims. We emphasize that the significance level when rejecting the null hypothesis can lead to the incorrect conclusion that the H_{0} hypothesis is unlikely to occur at that significance level. We demonstrate that the significance level is unrelated to the posterior probability of H_{0}, given the observed data set, and that this posterior probability is very much higher than implied by the significance level.
Methods. We use Bayes theorem to derive the posterior probability of that H_{0} is true assuming an alternative hypothesis H_{1} that a mode is present, taking some prior for the mode height, mode amplitude and linewidth.
Results. We compute the posterior probability of H_{0} for the p modes detected on HD 49 933 by CoRoT.
Conclusions. We conclude that the posterior probability of H_{0} provide a much more conservative quantification of the mode detection than the significance level. This framework can be applied to any similar stellar power spectra obtained to complete asteroseismology.
Key words: methods: analytical  methods: statistical  Sun: helioseismology
1 Introduction
In the field of helioseismology, the null hypothesis H_{0} was used by Appourchaux et al. (2000) to infer upper limits to the amplitude of g modes. The H_{0} hypothesis assumes that what is observed is pure noise. This hypothesis was applied to data of classical variable stars to detect peaks in their power spectra (Scargle 1982). In all of these papers, the authors assumed H_{0} and defined an upper limit corresponding to a threshold level of, e.g., 10%. Here we argue, that this threshold was arbitrarily chosen a priori. When accepting or rejecting the H_{0} hypothesis, there is no discussion whether borderline case should be rejected or accepted. This abrupt truncation between good and bad leads to a decision that could have been different if the threshold had been different.In the following sections, we first lay the foundations for understanding the meaning of the H_{0} hypothesis, and then explain what is commonly misunderstood about the H_{0} hypothesis. We then derive, for specific cases encountered in helio and asteroseismology, the posterior probability of H_{0}. We show how one can apply the formalism for real data, such as that gathered by the CoRoT mission for HD 49 933 and then present our conclusions.
2 Significance level and the H_{0} hypothesis
Fisher (1925) devised the well known Fisher test for testing the null hypothesis (H_{0}). In this test, a threshold of 5% is commonly used and the pvalue quoted is the value of the test if it is less than this threshold. For instance, a result of 4.9% would result in rejecting the H_{0} hypothesis, while 5.1% would result in accepting the H_{0} hypothesis. When the H_{0} hypothesis is rejected, the reported pvalue is used as a significance level for the validating or not accepting the H_{0} hypothesis. In the medical field, the socalled borderline cases led to findings related to effectiveness of medicine that were, sometimes, not proven by subsequent studies. The controversy about the use of pvalues that occurred in the medical world is directly related to the abrupt and arbitrary cutoff of the threshold applied (be 5% or 10%). Although the relevance of what has been found in the medical field could seem remote to most astrophysicists, it is extremely relevant to understand that the improper use of pvalues is the same as the improper use of the socalled significance level.
Reporting a small number for the significance level should not be used to claim the proper rejection of the H_{0} hypothesis. The mistake is to ascribe a significance level to a measurement carried out only once, not repeated and spanning just a very small volume of the space of the parameters. When making an observation of a random variable x, one wants to check the probability that what is observed could be due to noise. For that purpose, a test statistic is derived called T(x). If one observes a value of x=X that would not reject the H_{0}, then one can compute, p, the significance level or pvalue, defined to be:
(1) 
where P_{0} is the probability that when H_{0} is true. The test set by Eq. (1) is to confirm that the statistical test T(x) has been compared with the value given by T(X). When computing the statistical test T(x), one does not span the space of X: one has x=X which is the socalled point null hypothesis. In other terms, what is the probability that one has exactly that value of x? This is a completely different question from knowing how true H_{0} is, i.e., obtaining . Berger & Sellke (1987) provided a way of deriving with respect to and , where H_{1} is the alternative hypothesis (i.e., there is a signal). Using Bayes theorem, they obtained the socalled posterior probability of H_{0} given the observed data x:
(2) 
where and are the probabilities related to the H_{0} and H_{1} hypothesis, respectively. We note that we would like to determine , the probability of H_{0} being true given the data we have, not the probability of the data x given H_{0}, i.e., . To minimize the impact of the probabilities assigned to both hypotheses, we assume that these are equally probable (p_{0}=0.5):
(3) 
Based under these assumptions, Sellke et al. (2001) found that the probability that H_{0} is true given some observed data x of a random variable X has a lower bound
(4) 
An immediate consequence is that for a significance level of 1%, the odds against H_{0} are at least 10 to 1, and for 10%, the odds against H_{0} are at least 2.6 to 1. In both cases, the likelihood of incorrectly rejecting H_{0} is much higher than inferred by the pvalue, by at most a factor of 10 and 4, respectively.
Sellke et al. (2001) were able to set a lower value to for almost an arbitrary alternative H_{1} hypothesis. We show that there is indeed a lower bound when one wants to detect peaks in power spectra. Hereafter, we give several examples of how one can derive in practice the posterior probability of H_{0}.
3 Posterior probability for peak detection
Here we provide analytical examples of how one can obtain the posterior probability of H_{0} for some cases encountered in astero and helioseismology.
3.1 Longlived modes
3.1.1 Mode height known a priori
We search a peak restricted to a single frequency bin in a power spectrum which has a
with 2 d.o.f. statistics, for which a bin has reached a value x. We wish to verify whether this is caused by either a true sine wave or due to noise. We have for the H_{0} hypothesis,
(5) 
The noise is assumed to be 1. For the alternative hypothesis H_{1}, we assume that there is a signal of a longlived mode, i.e., restricted to one bin, for which the mode height H is known and the mode is stochastically excited (like a stellar p mode). We then have:
(6) 
Equation (3) is then rewritten for our problem as
(7) 
Since the significance , we finally have that
(8) 
It can be shown that the minimum of is reached for for the value:
(9) 
In this case, the lower bound defined by Eq. (4) is reached.
3.1.2 Mode height unknown
Most of the time one does not know of course the height of the mode to be detected. We can assume a prior for the mode height which can be e.g., a uniform distribution or a gaussian function. For example, if we assume that the mode height is uniformly distributed over some range [0, ], is rewritten as(10) 
Then Eq. (3) can be rewritten as
(11) 
It can be shown that a minimum is reached at a value of by solving
(12) 
The minimum is then given by
(13) 
From Eqs. (8) and (9), we can deduce that
(14) 
This means that when there is more uncertainty about the possible height of the mode, one is less likely to reject the hypothesis.
3.2 Shortlived modes
Appourchaux (2004) described how one can detect a mode having a lifetime shorter than the observation time. He suggested smoothing the power spectrum to increase the signaltonoise ratio and provided an analytical expression for H_{0} related to the summation over n bins of a statistic with constant mean, and for H_{1} related to the summation over n bins of a statistic with different means due to the presence of the mode profile. Therefore, when the spectrum is summed over n bins, we can derive for the H_{0} hypothesis(15) 
where the mean of the power spectrum is 1, while n is the mean of the smoothed power spectrum (for simplicity, we assumed that S=1), and is the Gamma function. The significance level x is then given by solving
Assuming that the mode has a known amplitude A and a known linewidth , we can write using Eq. (8) of Appourchaux (2004) the approximation
(17) 
where and , defined in Appourchaux (2004) are obtained by integrating symmetrically about the central frequency of the mode, thereby ensuring that the signal is maximum; and are functions of the mode height and . If we do not know the mode amplitude and its linewidth, we can specify a prior for the amplitude and the linewidth, which, for instance, can be done using a model of mode excitation. Here we assume that the mode amplitude A and the linewidth are independent (this would not be the case of H and ). If we use uniform priors, we then have
(18) 
This equation would be similar if we were to have a uniform prior on the mode height, where A would then be replaced by H. Substituting Eqs. (16) and (17) into Eq. (3), we can then obtain the posterior probability when the mode amplitude and linewidth are known. If we replace Eqs. (17) by (18), we can then obtain the probability when the mode amplitude and linewidth are not known.
Figure 1: Posterior probability as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) for a significance level of 10%. For the known mode height, the minimum is reached for an height of 1.3 with a value of 38%. For the unknown mode height, the minimum is reached at a value of 40%. Posterior probability as a function of the known height (dashed line), or as a function of the maximum mode height of the uniform prior (continuous line) for a significance level of 1%. For the known mode height, the minimum is reached for an height of 3.6 with a value of 11%. For the unknown mode height, the minimum is reached at a value of 12.7%. 

Open with DEXTER 
3.3 Discussion
Figure 1 shows the posterior probability for longlived modes of a known mode height (Eq. (8)) and an unknown mode height (Eq. (11)) for two different significance levels. Figure 2 shows the posterior probability for shortlived modes of a known mode height and linewidth (Eqs. (15) and (17)) and for an unknown mode height and linewidth (Eqs. (15) and (18), with H replacing A) for a single significance level; even a significance level of 1% does not provide a more robust rejection of the H_{0} hypothesis. These posterior probabilities have a lower bound which means that even a very low significance level is no guarantee for positive detection!It is also counterintuitive that the posterior probability increases when the mode height (known or unknown) increases. We recall that the significance level p corresponds to the level x at which the peak has been observed (i.e., it is for longlived modes). If we assume a priori that the mode has a high mode height, then the observation at a low significance level indicates that our assumption about the high mode height is incorrect, and that the data dismisses the a priori assumptions made about the mode height. In other words, it is more probable that the H_{0} hypothesis is true.
In the absence of an alternative hypothesis, it is advisable to define a low significance value that will probably reject the H_{0} hypothesis. Figure 3 shows the lower bound^{} set by Eq. (4) compared to the minimum found using Eq. (11) for the uniform prior on the mode height.
Figure 2: Posterior probability for a shortlived mode after smoothing the power spectra over 10 bins corresponding to a window twice as large as the mode linewidth. The posterior probability is given as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) and a uniform prior for the linewidth, for a significance level of 10%. 

Open with DEXTER 
Figure 3: Lower bound to the posterior probability as function of the significance level for the known mode height (dashed line) and for the uniform prior (solid line). 

Open with DEXTER 
4 Application to the CoRoT data: HD 49 933
The computation of the posterior probability was applied to CoRoT data for illustrative purposes. The data used are those of the first initial run performed on HD 49 933 (Appourchaux et al. 2008). The objective was to provide an objective way of detecting oscillation modes in HD 49 933 that could be applied to any other star.The methodology used for deriving the posterior probability was as follows:
 we computed the power spectrum from the detrended time series, as in Appourchaux et al. (2008);
 we smoothed the spectrum over n bins using a boxcar;
 we selected 30 50Hz wide windows starting at 1200 Hz (the 50Hz window corresponds roughly to half the large frequency separation);
 for each window, we computed the median in the window of the smoothed spectrum which provides an estimate of the mean noise level if the modes are not present;
 the smoothed spectrum was normalized in each window by dividing by the median and multiplying by the number of smoothing bins n which provides values commensurate with these of Eq. (15);
 we applied the H_{0} hypothesis for a detection probability that a signal caused by to noise of 10% over all the 30 windows, taking into account the fact that in each window the number of independent bins is 50 ( is the frequency resolution of the original power spectrum). The detection probability is then 0.1 per independent bin;
 we then solved Eq. (16) for given the detection level given above;
 in each window, we then selected the bins that are greater than , i.e., we accepted or rejected the H_{0} hypothesis;
 after the selection, we retained the greatest value found in the window corresponding to the central frequency of the mode (See Eq. (17));
 we then computed the posterior probability of H_{0} given by Eq. (3) using Eqs. (15) and (18) assuming some prior on the mode height and linewidth as described below;
 for comparison, we also computed the significance level as given by Eq. (16) from the value of .
Figure 4: ( Left) Theoretical mode linewidth of HD 49 933 as a function of frequency. ( Right) Theoretical mode amplitude of HD 49 933 as a function of frequency. 

Open with DEXTER 
Figure 5: ( Top) Comparison of the posterior probability (solid line) with the significance level (dashed line) as a function of frequency for a spectrum smoothed over 5 bins (Hz). ( Bottom) Posterior probability as a function of frequency for a spectrum smoothed over 5 bins (solid line) and over 50 bins (dashed line), of sizes Hz and Hz, respectively. 

Open with DEXTER 
The uniform priors for amplitude and linewidth are derived from the theory by taking into account an uncertainty factor in the theoretical model. For amplitude, we assumed that the maximum is larger than given in Fig. 4 (twice in energy); for linewidth, we assumed that the maximum is twice as great as that given in Fig. 4. We note that a larger prior increases the posterior probability, as shown on Figs. 1 and 2. We assumed that the noise floor in HD 49 933 is given by the photon noise, which is about 0.15 ppmHz (Appourchaux et al. 2008).
Figure 5 shows the results of the procedure described above. It is clear that the posterior probability is higher than the significance level, i.e., the posterior probability provides a moreconservative number (H_{0} more likely). The smoothing procedure also shows two effects that were predicted by Appourchaux (2004): first, short lived modes are easier to detect when the spectrum is smoothed, second, long lived modes are more difficult to detect when the spectrum is smoothed. The first effect manifests itself in the larger number of detected modes at higher frequency and by the decrease of the prior probability (i.e., the signal is more likely). The second effect is seen at low frequency where a couple of modes have their prior probability increased to nonnegligible value after smoothing (i.e., the signal is less likely). When we compare with modes reported by Appourchaux et al. (2008), we find that more than 85% of the l=02 mode pairs and l=1 modes are recovered. An additional mode at 2579 Hz is detected that could be an l=1 mode according to the identification of Appourchaux et al. (2008).
5 Conclusion
The significance level refers to the significance of the data given the hypothesis, while we are interested in the posterior probability of the null hypothesis given the data. Here we have shown that for a significance level of 10%, the posterior probability of the null hypothesis is at least 38% when there is no alternative hypothesis. We have illustrated how one can in practice calculate and compute the posterior probability for the null hypothesis. This has been applied to several theoretical examples and to the CoRoT data. For the first time, we have shown how one can assess the detectability of shortlived p modes in a power spectrum. The methodology can be applied to any stellar power spectrum for which theoretical expectations are available.
Acknowledgements
We are grateful to John Leibacher for checking the English and for correcting several typos. We are thankful to the referee for making the paper clearer.
References
 Appourchaux, T. 2004, A&A, 428, 1039 [NASA ADS] [CrossRef] [EDP Sciences] (In the text)
 Appourchaux, T., Fröhlich, C., Andersen, B., et al. 2000, ApJ, 538, 401 [NASA ADS] [CrossRef] (In the text)
 Appourchaux, T., Michel, E., Auvergne, M., et al. 2008, A&A, 488, 705 [CrossRef] [EDP Sciences] (In the text)
 Berger, J., & Sellke, T. 1987, J. Am. Stat. Assoc., 82(397), 112 (In the text)
 Dupret, M.A., Samadi, R., Grigahcene, A., Goupil, M.J., & Gabriel, M. 2006, Commun. Asteroseismol., 147, 85 [CrossRef] (In the text)
 Fisher, R. A. 1925, Statistical Methods for Research Workers (Edinburgh, Scotland: Oliver and Boyd), 299 (In the text)
 Grigahcène, A., Dupret, M.A., Gabriel, M., Garrido, R., & Scuflaire, R. 2005, A&A, 434, 1055 [NASA ADS] [CrossRef] [EDP Sciences] (In the text)
 Samadi, R., Ludwig, H.G., Belkacem, K., et al. 2009a, A&A, submitted (In the text)
 Samadi, R., Ludwig, H.G., Belkacem, K., Goupil, M., & Dupret, M.A. 2009b, A&A, submitted (In the text)
 Scargle, J. D. 1982, ApJ, 263, 835 [NASA ADS] [CrossRef] (In the text)
 Sellke, T., Bayarri, M. J., & Berger, J. 2001, The American Statistician, 55, 62 [CrossRef] (In the text)
Footnotes
 ... CoRoT^{}
 The CoRoT space mission, launched on 2006 December 27, was developed and is operated by the CNES, with participation of the Science Programs of ESA, ESA's RSSD, Austria, Belgium, Brazil, Germany and Spain.
 ... bound^{}
 The same lower bound as Eq. (9) for a known mode height.
All Figures
Figure 1: Posterior probability as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) for a significance level of 10%. For the known mode height, the minimum is reached for an height of 1.3 with a value of 38%. For the unknown mode height, the minimum is reached at a value of 40%. Posterior probability as a function of the known height (dashed line), or as a function of the maximum mode height of the uniform prior (continuous line) for a significance level of 1%. For the known mode height, the minimum is reached for an height of 3.6 with a value of 11%. For the unknown mode height, the minimum is reached at a value of 12.7%. 

Open with DEXTER  
In the text 
Figure 2: Posterior probability for a shortlived mode after smoothing the power spectra over 10 bins corresponding to a window twice as large as the mode linewidth. The posterior probability is given as a function of the known mode height (dashed line), or as a function of the mode height range () of the uniform prior (continuous line) and a uniform prior for the linewidth, for a significance level of 10%. 

Open with DEXTER  
In the text 
Figure 3: Lower bound to the posterior probability as function of the significance level for the known mode height (dashed line) and for the uniform prior (solid line). 

Open with DEXTER  
In the text 
Figure 4: ( Left) Theoretical mode linewidth of HD 49 933 as a function of frequency. ( Right) Theoretical mode amplitude of HD 49 933 as a function of frequency. 

Open with DEXTER  
In the text 
Figure 5: ( Top) Comparison of the posterior probability (solid line) with the significance level (dashed line) as a function of frequency for a spectrum smoothed over 5 bins (Hz). ( Bottom) Posterior probability as a function of frequency for a spectrum smoothed over 5 bins (solid line) and over 50 bins (dashed line), of sizes Hz and Hz, respectively. 

Open with DEXTER  
In the text 
Copyright ESO 2009
Current usage metrics show cumulative count of Article Views (fulltext article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 4896 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.