Online material

Appendix A: Sampling variance of weighted mean square error

If we assume the prediction errors are distributed normally with total variance, Eq. (5): , then the sampling variance of the weighted mean square error is (A.1)The standard error on WRMS is then .

Appendix B: Maximum likelihood estimators for intrinsic prediction error and covariance

Let the intrinsic variance of predictions () from method P,Q be . Denote the intrinsic covariance between distance prediction errors from P and Q as c_PQ. The intrinsic correlation is ρ_PQ = c_PQ/(σ_Pσ_Q). Let θ = (σ_P,σ_Q,ρ_PQ) be the vector of this triplet of quantities. These parameters can be arranged in an intrinsic covariance matrix: (B.1)We wish to estimate them from the cross-validated predictions. We derive estimators for this intrinsic covariance using maximum likelihood, and also derive its uncertainty.

We assume that the pair of prediction errors are jointly distributed normally around zero with total covariance (B.2)The measurement error covariance matrix, , contains the measurement variances, , for model P and model Q, on the diagonal, and any covariance due to observational error in the off-diagonal. Since supernova s is subject to the same random peculiar velocity under both models P and Q, the covariance from peculiar velocity dispersion is (B.3)The negative log likelihood for the unknown (σ_P,σ_Q,ρ_PQ), given the set of distance predictions is (B.4)We numerically maximize the likelihood with the constraints σ_P,σ_Q > 0 and |ρ_PQ| < 1. Once we have found the maximum likelihood estimate (MLE) , we can compute its error by

numerically evaluating the Hessian of the negative log likelihood, ). The sampling covariance (error) of the MLE is estimated from the Fisher information: . The standard errors in each of (σ_P,σ_Q,ρ_PQ) are the square roots of the diagonal elements of . The off-diagonal elements contain the estimation covariance between the three parameters. If the difference in intrinsic prediction error between the two models is Δ = σ_P − σ_Q, the sampling variance of Δ is (B.5)where is the (i,j) element of the error covariance matrix of the MLE. This error estimate accounts for covariance from random peculiar velocities and the intrinsic correlation between two models. Notably, a large |ρ_PQ| will affect the significance of the difference, Δ.

From the prediction errors of a single method,  {Δμ_s} , we can estimate the rms intrinsic prediction error σ_pred. The negative log likelihood simplifies to (B.6)The maximum likelihood estimate is found by minimizing this or finding the zero of the score function . If N is large enough, the standard error on can be estimated using the Fisher information at the MLE: (B.7)An estimate of the sampling variance of the maximum likelihood estimate of the intrinsic variance is the inverse of the Fisher information . The standard error of itself is the square root of . This estimate of the intrinsic dispersion “subtracts” out the contribution of random peculiar velocities and measurement error to the total dispersion.

Appendix C: Results for other spectroscopic indicators at maximum light

We present our results using the absorption velocity (v_abs), the full-width at half-maximum (FWHM), the relative absorption depth (d_abs), the pseudo-equivalent width (pEW), and the various spectroscopic ratios ℛ(X) [Eqs. (16)–(22)] in Tables C.1–C.5.

© ESO, 2010