© ESO, 2016

1. Introduction: a summary of the HFI pipeline

This paper, one of a set associated with the 2015 Planck¹ data release, is the first of two that describe the processing of the data from the High Frequency Instrument (HFI). The HFI is one of the two instruments on board Planck, the European Space Agency’s mission dedicated to precision measurements of the cosmic microwave background (CMB). The HFI uses cold optics (at 4 K, 1.6 K, and 100 mK), filters, and 52 bolometers cooled to 100 mK. Coupled to the Planck telescope, it enables us to map the continuum emission of the sky in intensity and polarization at frequencies of 100, 143, 217, and 353 GHz, and in intensity at 545 and 857 GHz. Paper A (this paper) describes the processing of the data at the time-ordered level and the measurement of the beam. Paper B (Planck Collaboration VIII 2016) describes the HFI photometric calibration and map making.

The HFI data processing for this release is very similar to that used for the 2013 release (Planck Collaboration VI 2014). Figure 1 provides a summary of the main steps used in the processing, from raw data to frequency maps both in temperature and polarization.

First, the telemetry data are converted to time ordered information (TOI). The TOI consists of voltage measurements sampled at 180.3737 Hz for each of the 52 bolometers, two dark bolometers, 16 thermometers, and two devices (a resistance and a capacitance) that comprise the HFI detector set. The TOI is then corrected to account for nonlinearity in the analogue-to-digital conversion (see Sect. 2). Glitches (cosmic ray impacts on the bolometers) are then detected, their immediate effects (data around the maximum) are flagged, and their tails are subtracted. A baseline is computed in order to demodulate the AC-biased TOI. A second-order polynomial correction is applied to the demodulated TOI to linearize the bolometer response. The minute-scale temperature fluctuations of the 100 mK stage are subtracted from the TOI using a combination of the TOI from the two dark bolometers. Sharp lines in the temporal power spectrum of the TOI from the influence of the helium Joule-Thomson (⁴He-JT) cooler (hereafter called 4-K lines) are removed with interpolation in the Fourier domain (see Sect. 3.3). The finite bolometer time responses are deconvolved from the TOI, also in the Fourier domain. For this release, the time response consists of four to seven thermal time constants for each bolometer. Several criteria based on statistical properties of the noise are used to reject the stable pointing periods (hereafter called rings) that are non-stationary (see Table 1). A subtractive jump correction is applied; it typically affects less than 1% of the rings and the amplitude of the jumps exceeds a tenth of the TOI rms in less than 0.1% of the rings.

At this point, the TOI is cleaned but not yet calibrated. The beam is measured using a combination of planet observations for the main beam and GRASP physical optics calculations² for the sidelobes (see Sect. 4). The focal plane geometry, or the relative position of bolometers in the sky, is deduced from Mars observations. The 545 and 857 GHz channels are photometrically calibrated using the response to Uranus and Neptune. The lower frequency channels (100, 143, 217, and 353 GHz, called the “CMB” channels) are calibrated with the orbital CMB dipole (i.e., the dipole induced by the motion of the Lagrange point L2 around the Sun).

In this paper, we describe the changes made to the processing since the 2011 and 2013 papers (Planck HFI Core Team 2011b; Planck Collaboration VI 2014). Section 2 gives a view of a step that has been added to the beginning of the pipeline, namely the correction for analogue-to-digital converter (ADC) nonlinearities. This step proves to be very important for the quality of the CMB data, especially at low multipoles. Section 3 deals with the addition of more long time constants in the bolometer response and other TOI. Section 4 presents refined beam measurements and models. Some consistency checks are reported in Sect. 5. The public HFI data products are described in Appendix A.

Paper B describes the new scheme for the CMB dipole calibration (the Planck orbital dipole calibration is used for the first time) and the submillimetre calibration on planets. Paper B also describes the polarized map making, including the derivation of far sidelobes, and zodiacal maps, as well as polarization correction maps due to bandpass mismatch: a generalized least-squares destriper is used to produce maps of the temperature and the two linear polarization Stokes parameters Q and U. About 3000 maps are obtained by splitting the HFI data into different subsets by, e.g., time period or detector sets. Consistency checks are performed in order to assess the fidelity of the maps.

2. ADC correction

Planck Collaboration VI (2014) reported that the HFI raw data show apparent gain variations with time of up to 2% due to nonlinearities in the HFI readout chain. In the 2013 data release (Planck Collaboration VIII 2014) a correction for this systematic error was applied as an apparent gain variation at the map making stage. The 2013 maps relied on an effective gain correction based on the consistency constraints from the reconstructed sky maps, which proved to be sufficient for the cosmological analysis.

For the 2015 data release we have implemented a direct ADC correction in the TOI. In this section, we describe the ADC effect and its correction, and its validation through end-to-end simulations (see also Sect. 5.4). Internal checks of residual ADC nonlinearity are shown in Sect. 5.

2.1. The ADC systematic error

The HFI bolometer electronic readout (Planck Collaboration 2015) includes a 16-bit ADC, the flight-qualified version of Maxwell Technologies model 7809LP having a very loose tolerance on the differential nonlinearity (the maximum deviation from one least significant bit, LSB, between two consecutive levels, over the whole range), specified to be not worse than one LSB. The implications of this feature for HFI performance had not been anticipated, and it did not produce any detected effect in ground test data. However, it proved to be a major systematic effect impacting the flight data. A wide dynamic range at the ADC input was needed to both measure the CMB and the sky foregrounds, and properly characterize and remove the tails of glitches from cosmic-rays. Operating HFI electronics with the necessary low gains increased the effects of the ADC scale errors on CMB data.

We have developed a method that reduces the ADC effect on the angular power spectra by more than a factor of 10 for most bolometers. There are three main difficulties in making this correction.

• The chip linearity defects were characterized with insufficientprecision before launch. We have designed and run a specificcampaign to map the ADC nonlinearity from flight data(Sect. 2.2).

• Each HFI data sample in the TOI is the sum of 40 consecutive ADC fast samples, corresponding to half of the modulation cycle. The full bandwidth of the digital signal is not transmitted to the ground. The effective correction of TOI samples due to ADC defects requires the knowledge of the shape of the fully-sampled raw data at the ADC input. A shape model is built from the subset of fully-sampled downloaded data, transmitted to the ground at a low rate (80 successive fast samples every 101.4 s for every detector), hereafter called “the fully-sampled raw data”.

• The ⁴He-JT 4-K cooler lines in the TOI (Sect. 3.3) result from a complex parasitic coupling. Much of this parasitic signal is a sum of 20 Hz harmonics, synchronous with the readout clock, with nine slow sample periods fitting exactly into two parasitic periods. Capture of a sequence of 360 raw signal samples would have allowed a direct reconstruction of the full patterns of this parasitic signal. However, the short downloaded sequences of 80 samples always fall in the same 4-K phase range, allowing us to cover only 2/9 of the full pattern. To properly model the signal at the ADC input, one must decipher this parasitic signal over its full phase range. The present model relies both on the full sampling subset and on the 4-K lines measured in the TOI (see Sect. 3.3 and references therein).

The signal model, including the 4-K line parasitic part, is described in Sect. 2.3.

2.2. Mapping the ADC defects

The defects of an ADC chip are fully characterized by the input levels corresponding to the transitions between two consecutive output values (known as digital output code, or DOC). An ADC defect mapping is usually run on a dedicated ground test bench. We made this measurement on two spare flight chips to understand the typical behaviour of the circuit in the range relevant to the flight data. These measurements revealed a 64-DOC periodic pattern, precisely followed by most of the DOC, except at the chunk boundaries, which allowed us to build a first, approximate defect model with a reduced number of parameters (Planck Collaboration VIII 2014). Such behaviour is understood from the circuit design, where the lowest bits come from the same components over the full ADC scale. However, because each ADC has a unique defect pattern, data from these ground tests could not be used directly to correct the flight data.

The parameters for the on-orbit chips were extracted for each HFI bolometer using data samples of the thermal signal, effectively a Gaussian noise input. These ADC-dedicated flight data, herafter called “warm data”, were recorded during the 1.5 years of the LFI extended mission, between February 2012 and August 2013. During this period the bolometer temperature was stable at about 4 K, a temperature at which the bolometer impedance is low, giving no input signal or parasitic pickup on the ADCs apart from a tunable offset and Gaussian noise with rms values around 20 equivalent mean LSBs. The defect mapping was obtained by inverting the histograms of the accumulated fully-sampled raw data, as explained below.

The warm data consist of large sets of n_tot counts of 80 sample raw periods taken in stable conditions with different bias currents and/or compensation voltages on the input stage tuned to mostly sample the central area of the ADC. We denote the signal value at the ADC input ν, and ν_i is the input level corresponding to the transition between DOC (i − 1) and DOC i. We call the probability density function of the kth sample of the 80 samples p^k(ν). This is assumed to be a Gaussian distribution with a mean depending on k and on the data set, and a variance σ² depending only on the data set.

Every set k is a series of { n_i } histogram bin statistics, following a multinomial distribution. The mean values, , are linked to p^k and { ν_i } by the following set of equations: (1)A maximum likelihood technique maps the to the { n_i }. To extract { ν_i } from Eq. (1) one needs an estimate of and σ². The equations are solved by recurrence, starting from the most populated bin. Since the solution depends on and σ², normally one would need to know their true value. In fact, this system has specific properties that allow recovery of the correct ADC scale without any other information.

• Its solution { ν_i } is extremely sensitive to . An incorrect input value gives unphysically diverging ADC step sizes; this allows us to choose the that gives the most stable solution. This prescription makes the { ν_i } solution in the populated part of the histrogram independent from the hypothesis on .

• The step size solution is proportional to σ, so the solutions for different data sets can be intercalibrated to fit a common ADC scale, which allows the unknown σ to be eliminated, provided all 80 samples of a data set have the same noise value.

Figure 2 shows a typical example of the maximum likelihood result, before applying the periodic feature constraints on the solution. This method is based on the strong assumption that the noise is Gaussian. Some data showing small temperature drifts have not been used. With this method, the ADC step sizes are nearly independently measured, so there is a random-walk type of error on the DOC positions, which is limited by the 64 DOC periodicity constraint. The 64 DOC pattern is obtained from the weighted average of the values obtained in the ± 512 DOC around mid-scale, except for the first 64 DOC above mid-scale, which do not follow the same pattern.

The main step is found at mid-scale. For all channels, there are more than 10⁵ samples per DOC histogram bin in the small ±512 output code range around mid-scale that is explored by flight data. Outside this region, the smaller numbers of samples lead to bigger drifts in the likelihood solution.

Both residual distributions and simulations give an estimate of the precision of the present defect recovery below 0.03 LSB for any DOC over this range. Systematic errors on large DOC distances, not taken into account in this model, are smaller than 0.2 LSB over a range of 512 LSB. At this level, we see residuals due to violation of the rms noise (σ) stability assumption, and we are working on an improved version of the ADC defect recovery to be included in future data releases.

2.3. Input signal model

The signal at the input of the ADCs is the sum of several components, including the modulation of the bolometer, noise from the bolometer, and the electronics, and bolometer voltage changes due to the astrophysical signal. The largest in amplitude is the modulation of the bolometer voltage with the combination of a triangle- and a square-wave signal, fed through bias capacitors. Most of the modulation is balanced (i.e., the voltage amplitude is reduced to nearly zero) by subtracting a compensating square-wave signal prior to digitization.

Spurious components also exist, such as the 4-K parasitic signal and various electronic leakage effects. Both are well described by slowly varying periodic patterns, assumed to be stable on a one-hour time scale.

The signal model used here is based on the linearity of the bolometer chain. It is a steady-state approximation of the signal shape produced by constant optical power on the bolometer. It is given by (2)where P is proportional to the sky signal, and the raw gain G_raw and the offset O are periodic functions of time. The raw gain period is the same length as the readout period, and the offset period is equivalent to the 4-K cooler period.

The parameters of the model have been extracted and checked over the whole mission from a clean subset of fully-sampled raw data (particle glitches and planet-crossings excluded). For each bolometer, G_raw is given by a set of 80 numbers, assumed to be stable over the full mission. The offsets are given ring by ring as sets of 360 numbers.

Figure 3 shows an example of the raw gain G_raw, for bolometer 100-1a. This function represents the fast time scale voltage across the bolometer. The detailed shape is due to the modulating bias current and to well-understood filters acting on the signal in the readout chain (for details, see Sect. 4.1 of Lamarre et al. 2010). The signal model (Eq. (2)) assumes that every pair of telemetered fast data samples is the integral of one half of this function, linearly scaled by the input sky signal.

Figure 4 shows the evolution of the constant term over the mission, for the 143-6 bolometer, due to the spurious signals in the fully-sampled raw data window.

Electronics leakage is readout-synchronous and is therefore taken into account by the present model. This is not the case for the 4-K cooler parasitic signal. The available information in the TOI does not allow reconstruction of the shape of the 4-K line at the ADC input without ambiguities. Fortunately, this parasitic signal is, in most cases, dominated by a few harmonics that we constrain from the combined analysis of the constant term in the fully-sampled raw data and the 4-K-folded harmonics in the TOI. A constrained χ² method is used to extract amplitudes and phases ring per ring for the 20, 160, and 200 Hz components. Only these three harmonics are used in the present model. The effect of this approximation is checked by using both internal and global tests. Improved 4-K harmonic estimators will be included in future data releases.

2.4. ADC correction functions

The input shape model from Eq. (2) is used, along with the ADC DOC patterns and the S_phase value, to compute, ring by ring, the error induced by the ADC as a function of the summed 40 samples of the TOI. A set of input power values { P } is used to simulate TOI data: first,with the input model that includes the 4-K line model and the ADC non linearity model; and, second, with a parasite-free input model and a linear ADC. We identify real TOI data with the first result and draw from these simulations the correction functions that are then applied to the real data.

The combination of the two readout parities³ with different possible phases relative to the 4-K parasitic signal produces 18 different correction functions that are determined for each data ring and applied to every TOI sample. Figure 5 shows an example.

2.5. Simulation of the ADC effect

The measured ADC defects are included in the HFI end-to-end simulations (Sect. 5.4), run at the fully-sampled raw data level. This allows us to assess some consequences of the ADC nonlinearity on the data. For instance, the gain variations are well-reproduced. Figure 6 shows an example of survey-difference angular power spectra. The systematic effect coming from the ADC behaves as 1 /ℓ² at low ℓ, and is well suppressed by the ADC correction. This is an important check on the consistency of the ADC processing.

3. TOI processing

After the ADC correction, the software modules used to process the 2015 TOI are identical to those used for 2013, up to the stage of clean TOIs ready for selection, calibration, and map making (see Fig. 1). However, some of the modules’ input parameters have been fine-tuned to better control residual systematic errors present in the 2013 data. We describe these tunings here, and later assess their accuracy in Sect. 5.

3.1. Pointing and focal-plane reconstruction

Satellite attitude reconstruction is the same for both Planck instruments in the 2015 release and is described in the mission overview paper (Planck Collaboration I 2016). The major improvement since 2013 is the use of solar distance and radiometer electronics box assembly (REBA) thermometry as pointing error templates that are fitted and corrected.

As in Planck Collaboration VI (2014), we determine the location of the HFI detectors relative to the satellite boresight using bright planet observations. Specifically, we use the first observation of Mars to define the nominal location of each detector. These locations do not correspond exactly to the physical geometry of the focal plane, since they include any relative shift induced by imperfect deconvolution of the time-response of the detectors, as described in Sect. 3.4.2. After this initial geometrical calibration, the reconstruction is monitored by subsequent planet observations. We find that the detector positions are stable to < 10′′ with an rms measurement error of about 1′′.

3.2. Cosmic ray deglitching

The deglitching method, described in Planck Collaboration X (2014), consists of flagging the main part (with S/N> 3.3) of the response to each cosmic ray hit and subtracting a tail computed from a template for the remaining part. The flagged part is not used in the maps. The method and parameters are unchanged since 2013. As described by Planck Collaboration X (2014) and Catalano et al. (2014), three main populations of glitches have been identified: (i) short glitches (with a peaked amplitude distribution), due to the direct impact of a cosmic particle on the grid or the thermometer; (ii) long glitches, the dominant population, due to the impact of a cosmic particle on the silicon die, or support structure of the bolometer’s absorber; (iii) and slow glitches, with a tail similar to the long ones and showing no fast part. The physical origin of this last population is not yet understood.

The polarization-sensitive bolometers (PSBs) are paired, with two bolometers (called a and b) sharing the same housing (Jones et al. 2003). The dies are thus superimposed and most of the long glitches seen in one detector are also seen in the other. A flag is computed from the sum of the a and b signal-subtracted TOI after each has been deglitched individually. This new flag is then included in the total flag used for both the a and b bolometers.

For strong signals, the deglitcher threshold is auto-adjusted to cope with source noise, due to the small pointing drift during a ring. Thus, more glitches are left in data in the vicinity of bright sources (such as the Galactic centre) than elsewhere. To mitigate this effect near bright planets, we flag and interpolate over the signal at the planet location prior to the TOI processing. While a simple linear interpolation was applied in the first release (Planck Collaboration XII 2014), an estimate of the background signal based on the sky map is now used to replace these samples. For the 2015 release, this is done for Jupiter in all HFI frequency bands, for Saturn at ν ≥ 217 GHz, and for Mars at ν ≥ 353 GHz.

Nevertheless, for beam and calibration studies (see Sect. 4 and Paper B), the TOI of all planet crossings, including the planet signals, is needed at all frequencies. Hence, a separate data reduction is done in parallel for those pointing periods and bolometers. For this special production, the quality of the deglitching has been improved with respect to the 2013 data analysis (see Appendix B).

3.3. ⁴He-JT cooler pickup and ring selection

Planck scans a given ring on the sky for roughly 45 min before moving on to the next ring (Planck Collaboration I 2014). The data between these rings, taken while the spacecraft spin-axis is moving, are discarded as “unstable”. The data taken during the intervening “stable” periods are subjected to a number of statistical tests to decide whether they should be flagged as unusable (Planck Collaboration VI 2014). This procedure continues to be adopted for the present data release. Here we describe an additional selection process introduced to mitigate the effect of the 4-K lines on the data.

The ⁴He-JT cooler is the only moving part on the spacecraft. It is driven at 40 Hz, synchronously with the HFI data acquisition. Electromagnetic and microphonic interference from the cooler reaches the readout boxes and wires in the warm service module of the spacecraft and appears in the HFI data as a set of very narrow lines at multiples of 10 Hz and at 17 Hz (Planck Collaboration VI 2014). The subtraction scheme used for the 2013 release, used here as well, is based on measuring the Fourier coefficients of these lines and interpolating them for the rings that have one of the harmonics of their spin frequency very close to a line frequency – a so-called “resonant” ring. However, it was noticed that the amplitude of the lines increased in the last two surveys. Therefore, as a precautionary step, resonant rings with an expected line amplitude above a certain threshold are now discarded.

In contrast to the other lines, the 30 Hz line signal is correlated across bolometers (see Fig. 7). It is therefore likely that the 4-K line removal procedure leaves correlated residuals on the 30 Hz line. The consequence of this correlation is that the cross-power spectra between different detectors can show excess noise at multipoles around ℓ ≃ 1800⁴ (see the discussion in Sect. 1 of Planck Collaboration XVI 2014 and in Sect. 7 of Planck Collaboration XV 2014). An example, computed with the anafast code in the HEALPix package (Górski et al. 2005), is shown in Fig. 8. To mitigate this effect, we discard all 30 Hz resonant rings for the 16 bolometers between 100 and 353 GHz for which the median average of the 30 Hz line amplitude is above 10 aW. Doing so, the ℓ = 1800 feature disappears. This issue is also addressed in Sect. 3.1 of Planck Collaboration XIII (2016).

Figure 9 shows the fraction of discarded samples for each detector over the full mission. It gathers the flags at the sample level, which are mainly due to glitches and the depointing between rings. It also shows the flags at the ring level, which are mostly due to the 4-K lines, but are also due to solar flares, big manoeuvres, and end-of-life calibration sequences, which are common to all detectors. The main difference from the nominal mission, presented in the 2013 papers, appears in the fifth survey, which is somewhat disjointed, due to solar flares arising with the increased solar activity, and to special calibration sequences. The full cold Planck-HFI mission lasted 885 days, excluding the calibration and performance verification (CPV) period of 1.5 months. During this time, HFI data losses amount to 31%, the majority of which comes from glitch flagging, as shown in Table 1. The fraction of samples flagged due to solar system objects (SSO), jumps, and saturation (Planck Collaboration VIII 2014) is below 0.1%, and hence negligible.

3.4. Detector time response

As noted in Planck Collaboration VII (2014) and Planck HFI Core Team (2011a), the detector time response is a key calibration parameter for HFI. It describes the relation between the optical signal incident on the detectors and the output of the readout electronics. This relation is characterized by a gain, and a time shift, dependent on the temporal frequency of the incoming optical signal. As in previous releases, it is described by a linear complex transfer function in the frequency domain, which we call the time transfer function. This time transfer function must be used to deconvolve the data in order to correct the frequency-dependent time shift, which otherwise significantly distorts the sky signal. The deconvolution also restores the frequency dependent gain. It is worth noting that: (i) the deconvolution significantly reduces the long tail of the scanning beam; (ii) it also symmetrizes the time response, which allows us to combine surveys obtained by scanning in opposite directions; and (iii) given that the gain decreases with frequency, the deconvolution boosts the noise at high frequency, as can be seen in Figs. 23 and 26. In order to avoid unacceptably high noise in the highest temporal frequencies, a phaseless low-pass filter is applied, with the same recipe as in Sect. 2.5 of Planck Collaboration VII (2014). This process results in a slightly rising noise in the high frequency part of the noise power spectrum, in particular for the slowest 100 GHz detectors. This noise property is ignored in the map making process, which assumes white noise and low frequency noise. We note, however, that the 100 GHz bolometers data are not used for CMB analysis at smaller angular scales, due to their wider main beams.

For this release, the time transfer function is based on the same model as the previous release. As a function of the angular frequency ω, it is defined by (3)where F(ω) is the term associated with the bolometer response, and H′(ω;S_phase,τ_stray) is the analytic model of the electronics transfer function, whose detailed equations and parameters are given in Appendix A of Planck Collaboration VII (2014). The electronics term depends only on two parameters: the phase shift between the AC bias current and the readout sampling, encoded by the parameter S_phase; and the time constant τ_stray associated with the stray capacitance of the cables connecting the bolometers to the bias capacitors (the stray capacitance enters in the h₀ term in the series of filters reported in Table A.1 of Planck Collaboration VII 2014).

The bolometer time transfer function is described by the sum of five single-pole low-pass functions, each with a time constant τ_i and an associated amplitude a_i: (4)In this version, five time constants are used rather than four, and the values of the parameters have been measured from a different combination of data than used previously. The extra low-pass function and the consequent parameter updates are motivated by the discovery of a time delay between the measured CMB dipole and the expected one. This time delay, or phase shift, is interpreted as the effect of an extra low-pass function, not accounted for in the previous version of the deconvolution process. As we will see, some level of time delay in the dipole remains even after deconvolution of the five low-pass functions. This small residual can generate problems in gain fitting and dipole subtraction, and is treated more efficiently at the map making level (see Paper B).

In addition to the dipole shift, new information comes from the stacking of glitches induced by high energy particles hits. As described in Planck Collaboration X (2014), short glitches are due to direct interaction of particles with the bolometer grid or thermistor. The time response of short events is then representative of the response to photons. Short glitches show a long tail, if enough events are stacked together. It was observed that the signals obtained by stacking short glitches and by stacking Jupiter scans decay to zero with the same time constants (see Fig. 10). Since the short glitch stacking has the benefit of a much higher signal-to-noise ratio, it provides the most sensitive measurement of the longest time constants. Although the physical process of energy injection into the detector is different for microwave photons and energetic particles, the heat dissipation is expected to be the same. For this reason, it was decided to take only the time constants from the glitches, and not the associated amplitudes. The impact of the incomplete correction of the transfer function is discussed in Sect. 3.4.1.

In summary, the values of the bolometer time transfer function parameters, a_i and τ_i, are measured with the following logic:

• the two fastest time constants, τ₁ and τ₂, and the associated amplitudes a₁ and a₂, are unchanged with respect to the previous version, i.e., they are estimated from planet observations;

• the two longest time constants, τ₄ and τ₅, are estimated from short glitch stacking, together with the a₅/a₄ ratio;

• τ₃, a₃, and a₄ are fitted from Jupiter scans, keeping τ₁, a₁, τ₂, and a₂ fixed, while the value of a₅ is set to keep the same ratio a₅/a₄ as in the glitch data;

• a₅ is fitted from the CMB dipole time shift. It should be noted that the same dipole time shift can be obtained with different combinations of a₅ and τ₅ and for this reason, τ₅ is recovered from the short glitches, and only a₅ from the dipole time shift.

This process was used for all channels from 100 to 217 GHz. For the 353 GHz detectors, different processing was needed to avoid strong non-optical asymmetries in the recovered scanning beam (see Appendix C for details). For the submillimetre channels, 545 and 857 GHz, the time transfer function is identical to that of the 2013 release. The values of the time response parameters are reported in Table C.1. The model and parameters will be improved by continuing this activity for future releases.

3.4.1. Time response errors

The beam model is built from time-ordered data deconvolved by the time transfer function. For this reason, and considering the constant rotation rate of Planck, the measured scanning beam absorbs, to a large extent, mismatches between the adopted time transfer function and the true one (Planck Collaboration VII 2014). In this sense, errors and uncertainties in the time transfer function should not be propagated into an overall window function uncertainty, since the time response acts as an error-free regularization function. Biases and uncertainties are taken into account in the beam error budget.

The beam window function correction of HFI’s angular power spectrum will not work on time scales longer than than roughly 0.5 s, given the size of the scanning beam map and the constant scan rate. Simulations with varying time response parameters show that errors at frequencies above 2 Hz are absorbed in the scanning beam map and propagated correctly to the effective beam window function to better than the scanning beam errors. Errors on time scales longer than 0.5 s are not absorbed by the scanning beam map, but propagate into the shifted dipole measurement and the relative calibration error budget (see Paper B).

Additionally, the time response error on time scales longer than 0.5 s can be checked by comparing the relative amplitude of the first acoustic peak of CMB anisotropies between frequency bands. The main calibration of HFI is performed with the CMB dipole (appearing in the TOI at 0.016 Hz), while the first acoustic peak at ℓ ≈ 200 appears at 6 Hz. Table 3 in Planck Collaboration I (2016) shows that between 100 and 217 GHz the agreement is better than 0.3%.

3.4.2. Focal plane phase shift from fast Mars scan

In December 2011, Planck underwent a series of HFI end-of-life tests. Among these, a speed-up test was performed increasing the spin rate to 1.4 rpm from the nominal value of 1 rpm. The test was executed on 7–16 December 2011, and included an observation of Mars. Right after the test, a second Mars observation followed, at nominal speed. The main result of this test was the ability to set the real position of the detectors in the focal plane, which, when scanning at constant speed, is completely degenerate with a time shift between bolometer data and pointing data. This time shift was supposed to be zero. During the test, it was found that the deconvolved bolometer signals peaked at a different sky position for the nominal scans and for the fast scans. This discrepancy was solved by introducing a time-shift between bolometer readout and pointing data. This time-shift is not the same for all detectors, and ranges from 9.5 to 12.5 ms, with a corresponding position shift ranging from 3.́3 to 4.́5 in the scanning direction. Due to time constraints, it was not possible to observe Mars twice with the full focal plane, so it was decided to favour the CMB channels and the planet was observed with all the 100, 143, 217, and 353 GHz detectors. For the other detectors, we used the average time-shift of all the measured detectors. Notably, this detector-by-detector shift resulted in a better agreement of the position in the focal plane of the two PSBs in the same horn.

Working with fast spin-rate deconvolved data is complicated by aliasing effects. For this reason the fitting procedure followed a forward sense approach, by modelling the signal with a beam centred in the nominal position, then convolving the fast and nominal timelines with the time transfer function, and fitting the correct beam centres by comparing data and model for both fast and nominal spin-rates. For the same reason, a direct comparison of deconvolved timeline has not proven to provide better constraints on the time-response parameters.

4. Planets and main beam description

We follow the nomenclature of Planck Collaboration VII (2014), where the “scanning beam” is defined as the coupled response of the optical system, the deconvolved time response function, and the software low-pass filter applied to the data. The “effective beam” represents the averaging of signal due to the scanning of the telescope and mapmaking, and varies from pixel to pixel across the sky.

Here we redefine the “main beam” to be the scanning beam out to 100′ from the beam axis. The sidelobe structure at this radius is dominated by diffraction at the mirror edges and falls as ∝ θ^-3, where θ is the angle to the main beam axis. The main beam is used to compute the effective beam and the effective beam window function, which describes the filtering of sky signals.

The smearing of the main beam cannot be significantly reduced without boosting the high-frequency noise. The regularization function (a low-pass filter) chosen has approximately the same width as the instrumental transfer function. The deconvolution significantly reduces the long tail of the scanning beam (see Fig. 11). The deconvolution also produces a more symmetric time response, so residual “streaking” appears both ahead and behind the main beam, though ahead of the beam it is at a level of less than 10^-4 of the peak response.

The “far sidelobes” are defined as the response from θ> 5°, roughly the minimum in the optical response. The response begins to rise as a function of angle θ beyond this due to spillover. The far sidelobes are handled separately from the beam effects (see Sect. 4.6 for justification and Paper B for details).

Observations of planets are used to estimate the main beams and to calibrate the 545 and the 857 GHz channels. The main beam is needed to correct for the filtering of the CMB sky by the instrument, details of which can be found in Planck Collaboration VII (2014). There were several key changes in the reconstruction of the main beam since the 2013 data release, described in detail in Appendix B.

• The TOI from Saturn and Jupiter observations are merged prior toB-spline decomposition, taking into account residual pointingerrors and variable seasonal brightness. This is achieved bydetermining a scaling factor and a pointing offset by fitting theTOI to a template from a previous estimate of the scanning beams.We iterate the planet data treatment, updating the template withthe reconstructed scanning beam. The process converges in fiveiterations to an accurcay of better than 0.1% in the effective beam window function.

• Steep gradients in the signal close to the planet reduce the completeness of the standard glitch detection and subtraction procedure, so the planet timelines are deglitched a second time.

• The beam pipeline destripes the planet data, estimating a single baseline between 3° and 5° before the peak for each scanning circle. Baseline values are smoothed with a sliding window of 40 circles. The entire scanning circle is removed from the beam reconstruction if the statistic in the timeline region used to estimate the baseline is far from Gaussian.

• The main beam is now recontructed on a square grid that extends to a radius of 100′ from the centroid, as opposed to 40′ in the 2013 data release. The cutoff of 100′ was chosen so that a diffraction model of the beam at large angles from the centroid predicts that less than 5 × 10^-5 of the total solid angle is missing.

• The scanning beam is constructed by combining data from Saturn observations, Jupiter observations, and physical optics models using GRASP software.

• No apodization is applied to the scanning beam map.

The update of the time response deconvolved data has slightly changed the scanning beam, the effective beam solid angles, and the effective beam window functions.

4.1. Hybrid beam model

A portion of the near sidelobes was not accounted for in the effective beam window function of the 2013 data (Planck Collaboration VII 2014; Planck Collaboration XXXI 2014). To remedy this, the domain of the main beam reconstruction has been extended to 100′, with no apodization. Saturn data are used where they are signal-dominated. Where the signal-to-noise ratio of the Saturn data falls below 9, azimuthally binned Jupiter data are used. At larger angles, below the noise floor of the Jupiter data, we use a power law (∝ θ^-3), whose exponent is derived from GRASP, to extend the beam model to 100′. Figure 12 shows a diagram of the regions handled differently in the hybrid beam model.

A summary of the solid angles of the hybrid beams is shown in Table 2.

Figure 13 shows a contour plot of all the scanning beams referenced to the centre of the focal plane.

4.2. Effective beams and window functions

As described in Mitra et al. (2011) and Planck Collaboration VII (2014), the FEBeCoP code is used to compute the effective beam (the scanning beam averaged over the scanning history) and the FEBeCoP and Quickbeam codes are used to compute the effective beam window functions. Statistics of the effective beams are shown in Table 3.

Using Saturn and Jupiter to reconstruct the main beam introduces a small bias due to the large disc size of the planets as compared to Mars (see Fig. 8 of Planck Collaboration VII 2014). Additionally, Saturn’s ring system introduces a slight frequency-band dependence of the effective size of the planetary disc (Planck Collaboration XXXIV, in prep.). Because of Saturn’s small size relative to the HFI beams, for ℓ< 4000 the symmetric part of Saturn’s shape dominates the window function. There is a small variation with HFI band in the apparent mean size of Saturn due to the different ring system temperatures, ranging from 9.̋25 at 100 GHz to 10.̋2 at 857 GHz. Because this variation introduces a bias of less than 2 × 10^-5 in for ℓ ≤ 4000, we ignore it and use a correction derived for a mean 9.̋5 Saturn disc for all bands.

The two effective-beam codes handle temperature-to-polarization leakage in slightly different ways. The dominant leakage comes from differences in the scanning beams of the polarization-sensitive detectors (see Sect. 4.7). The Quickbeam code mimics the data model of the mapmaking code and assumes that every polarization-sensitive detector at a given frequency has the same beam shape, and thus produces a single effective beam window function. Temperature-to-polarization leakage is handled later as a set of parameters in the likelihood of the angular power spectra. The FEBeCoP code produces effective beam window functions for the polarized power spectra that account for differences in the main beam. However, these are computed as the average power leakage of a given sky signal from temperature to polarization. Hence, these polarized window functions are not strictly instrumental parameters, since they rely on an assumed fiducial temperature angular power spectrum.

4.3. Beam error budget

As in the 2013 release, the beam error budget is based on an eigenmode decomposition of the scatter in simulated planet observations. A reconstruction bias is estimated from the ensemble average of the simulations. We generate 100 simulations for each planet observation that include pointing uncertainty, cosmic ray glitches, and the measured noise spectrum. Simulated glitches are injected into the timeline with the correct energy spectrum and rate (Planck Collaboration X 2014) and are detected and removed using the deglitch algorithm. Noise realizations are derived as shown in Sect. 5.3.1. Pointing uncertainty is simulated by randomizing the pointing by 1.̋5 rms in each direction.

The improved signal-to-noise ratio compared to 2013 leads to smaller error bars; for instance, at ℓ = 1000 the uncertainties on are now (2.2,0.84,0.81) × 10^-4 for 100, 143, and 217 GHz frequency-averaged maps respectively, reduced from the previous uncertainties of (61,23,20) × 10^-4.

A “Reduced Instrument Model” (RIMO; see Appendix A) containing the effective B(ℓ) for temperature and polarization detector sets, for auto- and cross-spectra at 100 to 217 GHz, is included in the release, for a sky fraction of 100%. Another RIMO is provided for a sky fraction of 75%. They both contain the first five beam error eigenmodes and their covariance matrix, for the multipole ranges [ 0,ℓ_max ] with ℓ_max = 2000, 3000, 3000 at 100, 143, and 217 GHz respectively (instead of 2500, 3000, 4000 previously). These new ranges bracket more closely the ones expected to be used in the likelihood analyses, and ensure a better determination of the leading modes on the customized ranges.

As described in Appendix A7 of Planck Collaboration XV (2014), these beam window function uncertainty eigenmodes are used to build the C(ℓ) covariance matrix used in the high-ℓ angular power spectrum likelihood analysis. It was found that the beam errors are negligible compared to the other sources of uncertainty and have no noticeable impact on the values or associated errors of the cosmological parameters.

4.4. Consistency of beam reconstruction

To evaluate the accuracy and consistency of the beam reconstruction method, we have compared beams reconstructed using Mars for the main beam part instead of Saturn, and using data from Year 1 or Year 2 only. We compared the window functions obtained with these new beams to the reference ones; new Monte Carlo simulations were created to evaluate the corresponding error bars, and the results are shown in Figs. 14 and 15. Note that the effective beam window functions shown in these figures are not exactly weighted for the scan strategy, rather we plot them in the raster scan limit, where is defined as the sum over m of the . The eigenmodes were evaluated for these different data sets allowing us to compare them with the reference beam using a χ² analysis. The discrepancy between each data set and the reference beam is fitted using N_d.o.f. = 5 eigenmodes using ℓ_max =1200, 2000, and 2500, for the 100, 143, and 217 GHz channels, respectively. The χ² is then defined as: (5)where c_i is the fit coefficient for eigenvector i, with eigenvalue λ_i. For each data set, the p-value to exceed this χ² value for a χ² distribution with N_d.o.f. degrees of freedom is indicated in Table 4. A high p-value indicates that the given data set is consistent with the reference beam within the simulation-determined error eigenvectors, with 100% indicating perfect agreement.

We find excellent agreement between the yearly and nominal beams for the SWB bolometer channels. However, in order to obtain reasonable beam agreement for the PSB detector sets, we find that the yearly beam errors must be scaled by a factor of 4. We conclude that there is an unknown systematic error in the beam reconstruction that is not accounted for in the simulations used to estimate the Monte Carlo error bars. We assign a scaling factor of 4 to the error eigenmodes to account for this systematic uncertainty. The additional error scaling has a negligible effect on the cosmological parameters.

4.5. Colour correction of the beam shape

In general, the measured beam shape is a function of the spectral energy distribution (SED) of the measurement source. This is of particular concern, because we measure the beam on a source with a roughly Rayleigh-Jeans SED, yet we use the effective beam window function to correct the CMB.

Planck Collaboration VII (2014) described possible levels of this bias derived using GRASP calculations with the pre-launch telescope model (Maffei et al. 2010; Tauber et al. 2010). The pre-launch calculations did not agree well enough with the data to allow a direct application of the colour correction. A new telescope model based on flight data, and presented in a forthcoming paper, predicts beams that agree better with the data at 100–217 GHz, but show worse agreement at 353 GHz. The new model predicts a different beam shape colour correction, though at a similar order of magnitude to that shown in Planck Collaboration VII (2014). This work is ongoing and will be completed after the 2015 release.

We search for a possible signature of the colour-correction effect in the data. First, we check for consistency in the CMB angular power spectra derived from different detectors and frequency bands using the SMICA algorithm, in order to find discrepancies in beam shape and relative calibration of different data subsets using the CMB anisotropies (Planck Collaboration XI 2016). There are hints of differences that are orthogonal to the beam error eigenmodes and appear as changes in relative calibration, but the preferred changes are at the 0.1% level. Because this correction is of the same order of magnitude as the relative calibration uncertainty between detectors, we treat it as insignificant.

Second, we look at the relative calibration between detectors within a band for compact sources as a function of the source SED. In this method, any discrepancies due to solid angle variation with SED are degenerate with an error in the underlying bandpass. Additionally, because of the lack of bright, compact sources with red spectra, this method is limited in its signal-to-noise ratio. In the limit that any detected discrepancy is completely due to solid-angle variation with colour, at the level of 1% in solid angle we do not detect variations consistent with the physical optics predictions.

Given the lack of measurement of this effect at the current levels of uncertainty, as well as uncertainties in the modelling of the telescope, we note that this effect may be present at a small level in the data, but we do not attempt to correct for it or include it in the error budget.

4.6. Effective beam window function at large angular scales

The main beam derived from planet observations and GRASP modelling extends 100′ from the beam axis, so the effective beam window function does not correct the filtering of the sky signal on larger scales (approximately multipoles ℓ< 50). In practice, due to reflector and baffle spillover, the optical response of HFI extends across the entire sky (Tauber et al. 2010; Planck Collaboration XIV 2014). According to GRASP calculations, the far-sidelobe pattern (FSL) further than 5° from the beam centroid constitutes between 0.05% and 0.3% of the total solid angle. To first order, this is entirely described by a correction to the calibration for angular scales smaller than the dipole (Planck Collaboration XXXI 2014).

More correctly, the far-sidelobe beam (defined here as the optical response more than 5° from the beam axis) filters large-angular-scale sky signals in a way that is coupled with the scan history of the spacecraft, and given a perfect measurement of the far-sidelobe beam shape, an effective beam window function could be constructed for the large angular scale with the same procedures used for the main beam’s effective beam window function.

We rely entirely on GRASP calculations to determine the large-scale response of the instrument. These calculations are fitted to survey difference maps (see Planck Collaboration XXXIV, in prep.), where the single free parameter is the amplitude of the GRASP far-sidelobe model beam. Because of a combination of the low signal-to-noise ratio of the data and uncertainty in the GRASP model, the errors on the fitted amplitude are of the order of 100%. While ground-based measurements have given us confidence in our limits on total spillover, we note that there is a large uncertainty in the exact shape of the far sidelobes. Concerning a possible evolution of the far sidelobe during the mission, we note that the background, as measured by the total power on the bolometers, decreases then stabilizes as a function of time. This proves that dust deposition on the Planck mirrors is negligible. Hence, the far sidelobes are likely to be very stable.

Figure 16 shows the correction to the effective beam window function due to the best-fit far-sidelobe model. The error bars are from the amplitude error in the fit Planck Collaboration XXXIV (in prep.) and they show that the uncertainty is larger than the predicted beam window function correction except at the dipole scales (ℓ = 1), aside from a marginally significant, but negligibly small, bump at ℓ = 5. The filtering of the CMB signal with the far-sidelobe beam is thus insignificant and can be ignored as a component of the effective beam window function. A map-level correction of Galactic contamination pickup in the far-sidelobes is more appropriate. However, given the large uncertainties in the far-sidelobe model shape, we choose to neglect this correction, and note that residuals from Galactic pickup may be present in the map. The only far sidelobe effect that is corrected in the data is an overall calibration correction, as described in Paper B.

4.7. Cross-polar response and temperature-to-polarization leakage

The data model used for the HFI polarization reconstruction assumes that the entire cross-polar response of each PSB is due to the detector itself, and so the beam shape of the cross-polar response is exactly the same as the co-polar beam shape. The corrugated feed horns exhibit some internal cross-polar response (Maffei et al. 2010). GRASP simulations show that this cross-polar response is at the level of 0.1–0.5%, considerably smaller than the detector cross-polar response (2–5%). Simulations using FEBeCoP show that the effect of ignoring the cross-polar optical beam shape results in a smoothing of polarization power spectra equivalent to an additional 10–20′′ Gaussian, which we neglect. Additionally the data model assumes identical beams for every PSB used to reconstruct the polarization; this creates some temperature-to-polarization leakage. Main-beam leakage dominates over the differential time response tails. Figure 17 shows a simulation of CMB temperature leakage into polarization power spectra using FEBeCoP, given the measured scanning beams within a band.

5. Validation and consistency tests

Here we describe some of the tests that have been done to validate the quality of the cleaned TOI. Other tests at the map level are described in Paper B. First we discuss the impact of the ADC correction. Then we analyse how each step of the TOI processing pipeline can alter the resulting power spectra and also study the noise properties. Finally we estimate the filtering function of the TOI pipeline with end-to-end simulations.

5.1. ADC residuals

5.1.1. Gain consistency at ring level

Using the undeconvolved TOI at the ring level, we can monitor the quality of the ADC correction with respect to the stability of the gain. For that purpose we can measure the relative gain of parity plus (g₊) and parity minus (g₋) samples (alternating samples). The two parities sample a very different part of the ADC scale, so that a gain mismatch is a diagnostic of the ADC nonlinearity correction. During a ring, the sky signal from both parities should be almost identical. We thus correlate a phase-binned ring (PBR) made of parity plus with the average PBR (made of both parities) and obtain g₊. We similarly obtain g₋. The gain half-difference (g₊ − g₋) / 2 is shown as a function of the ring number in Fig. 18 for a representative selection of four bolometers. The improvement obtained with ADC correction is significant with a root-mean-square dispersion decreasing by a factor of 2 to 3. Only a handful of bolometers show some discrepancies at the 10^-3 level after the ADC correction, namely 143-3b, 217-5b, 217-7a, 217-8a, and 353-3a. This internal consistency test at the ring level is not sensitive to gain errors common to both parities. However, it agrees qualitatively with the overall Bogopix gain variations shown in Paper B.

5.1.2. Parity map spectra

Two independent sky maps are produced from the two TOI parities; the differences between these are again expected to capture the ADC residual effects. Figure 19 compares full-sky spectra derived from maps built on raw and ADC-corrected data. Glitch-flagged samples have been removed. The behaviour in 1 /ℓ² at low ℓ is similar to the behaviour of simulated data as shown in Sect. 2.5.

These low-ℓ residuals, although much improved with respect to the 2013 delivery, are not yet fully under control. At the time of the 2015 delivery, work is continung to estimate how this effect propagates into the cross-polarization spectra, and to give a coherent description of the different low-ℓ systematics still present in HFI 2015 data. For this reason, among others, it has been decided to postpone the release of HFI low ℓ polarized data.

5.2. Filtering effects

It is customary to evaluate the effects produced by the pipeline on the signal as a filtering function, depending on the considered angular scale. Here we evaluate whether or how each pipeline module (see Fig. 1) filters the signal.

• ADC correction. The ADC nonlinearities add a spuriouscomponent to the power spectra. Correcting for it removes thiscomponent, reducing the C_ℓ by an amount smaller than 1%, that depends on the channel, and is very flat in ℓ.

• Glitch removal/flagging. Using simulations, Planck Collaboration X (2014) have shown that there is no significant bias (<10^-4) induced on the signal by the glitch removal procedure, thanks to the joint estimation of glitch tail and sky signal. The sky signal is estimated at the ring level using spline interpolation in order to correct for signal variations within ring pixels. However, the pointing jitter, combined with a strong signal gradient, can influence the glitch detection rate, mostly around the Galactic plane and planets. We have made simulations in order to evaluate an equivalent filtering effect using the glitch module called despike (Planck Collaboration X 2014). For CMB channels, the impact is clearly negligible. For submillimetre channels, the signal (mostly the interstellar dust emission) is changed at the level of 10^-4.

• AC modulation baseline subtraction. This has no impact on the signal, since it affects the TOI at the modulation frequency, which is cut out at the end of the TOI processing.

• Thermal decorrelation. This involves subtracting a TOI that is filtered at the minute timescale. The computation of the offsets per ring (in the destriping during the map making process) absorbs the long-term variability (typically longer than an hour) but not the mid-term variability (between 1 min and 1 h). The impact of the thermal decorrelation is to improve the accuracy of the destriping offsets. It is therefore not a filtering effect. The maximal impact of the thermal decorrelation is estimated by comparing C_ℓ using data processed with and without the thermal decorrelation. In Fig. 20, we see that the cross-spectrum of data that are fully decorrelated in time (as in Year 1 × Year 2) shows differences less than 10% for ℓ< 2000. The very-high-ℓ behaviour reflects the noise level of the reference spectrum. Consistency tests in the likelihood paper (Planck Collaboration XI 2016) are powerful enough to assess that the thermal residuals are negligible.

  Fig. 20 Relative differences of the Cℓ, computed on 60% of the sky, using data processed with and without the thermal decorrelation. Note that the zero-level has been recomputed in all cases to allow the comparison to be made. The amplitudes of the negative values are shown by dotted lines.

  Fig. 21 Relative differences of the Cℓ, computed on 60% of the sky, using data analysed with and without removal of the 4-K line frequencies.

• 4-K line removal. The maximal impact is estimated by comparing C_ℓ using data analysed with and without the removal of the 4-K line frequencies (keeping the same definition of valid rings in both cases). In Fig. 21 we see that only two lines have a strong impact: the 10 Hz line influences the power spectrum at ℓ ≈ 600 and the 30 Hz at ℓ ≈ 1800. For the 10 Hz line, the impact on the power spectrum is only seen in the auto-spectrum while the 30 Hz line remains in cross-spectra between two bolometers or between half rings. When the cross-spectrum is computed using data that are fully decorrelated in time (as in Year 1 × Year 2), no residual is seen. Consistency tests in the likelihood paper are powerful for assessing that the 4-K line residuals are negligible, thanks to the extra ring discarding scheme described in Sect. 3.3.

• Nonlinearity correction. The impact of this correction on measurements of the CMB has been propagated through the whole pipeline. Figure 22 shows that the angular power spectrum is underestimated by less than 0.3% between ℓ = 20 and ℓ = 2000 if no correction is applied, while still keeping the same calibration coefficients. We estimate that the nonlinearity correction is accurate within a 10% uncertatinty. Thus, the uncertainty of the bolometer nonlinearity correction translates into a filtering function uncertainty below the 10^-3 level over all angular scales.

• The Fourier transform module does of course filter the TOI. It deconvolves the time transfer function (Sect. 3.4) but also low-pass filters the data (Planck Collaboration VII 2014). It is important for high temporal frequencies, but affects all frequencies at the 10^-4 level down to the spin frequency where the filter is set to 1. The filtering effect is captured in the effective beam for scales of up to 1–2°. However, there could remain an effective filtering effect for larger angular scales.

In order to test the global pipeline filtering effect that could be induced by the interaction of several systematic effects, we resort to complete simulations, described in detail in Sect. 5.4.

5.3. Noise analyses

5.3.1. Noise spectrum

Noise spectra are best studied at the ring level using the redundancy built into the scanning strategy. The method is described in Planck Collaboration VI (2014) and consists of removing the sky-signal ring average from the TOI for each ring. Figure 23 shows the amplitude spectral density for representative HFI bolometers. This is modelled in Sect. 5.3.3 for the purpose of providing accurate inputs for simulations. For bolometers with relatively long time constants (especially at 100 GHz), the electronic noise is boosted at high frequency.

5.3.2. Noise stationarity

The noise stationarity is best studied by measuring the rms deviation of a TOI sample at the ring level, at the end of the TOI processing, where the average ring signal has been subtracted. The statistics are computed for samples which are valid, far from the Galactic plane, and far from bright point sources, in order to avoid strong residual gradients (see Planck Collaboration 2013). The bias induced by the signal subtraction is corrected for; the average value (after calibration) over the mission is given in Table 5. This is compatible with the values quoted in Table 4 of Planck Collaboration VI (2014).

Figure 24 shows trends of the total noise versus ring number before (2013 release) and after the ADC correction (this release), and histograms of the noise. In most cases, there is a significant decrease in the relative width of the histogram when the ADC correction is included, though that decrease is more significant for the low frequency channels, and since the distributions are non-Gaussian, very large decreases (or increases) indicate a large change in the structure of the distribution. Nevertheless, there are several cases where a jump in the noise is removed by the ADC correction (143-2a, 143-4a, 353-2). Other noise jumps (mostly at the sub-percent level except for bolometer 353-3a) remain unexplained. The noise of some bolometers shows a linear trend with time, but at the sub-percent level. For most bolometers, the noise is stationary below the percent level.

We have investigated the origin of the jumps in the noise level (e.g., for bolometer 353-2 at ring 5000 in the lower panel of Fig. 24). Most of the cases are not due to the ADC corrections (an exception is the 143-4a bolometer in Fig. 24). The extra noise component has been analysed by computing power spectra for selected rings. This falls into two categories: (1) for some detectors including 353-3a (the strongest case), it corresponds to an increase of the white noise component; (2) for other detectors, the extra component in the power spectrum is a bump concentrated in the 0.1 to 1 Hz range, which could be a very low level of RTS not detected by the non-Gaussianity tests that are routinely performed (Planck Collaboration VI 2014). Note that the small changes of noise levels with time are not taken into account by the mapmaking process, because the loss of optimality can be neglected.

5.3.3. Noise modelling

Individual ring noise power specral densities (PSDs) suffer from realization noise and signal residuals. For the purpose of simulating realistic noise from the PSDs, they need to be regularized by fitting a smooth model to the noisy PSD. To this end, we have constructed a physically-motivated model of the noise PSD consisting of

• 1.

  a photon/phonon noise component that is subject to thebolometric transfer function (suppressed above10 Hz);

• 2.

  an electronic noise component that is only subject to the digital low-pass filtering (abrupt suppression close to the Nyquist frequency);

• 3.

  a 1 /f slope comprising the low-frequency thermal fluctuations, glitch residuals, and other low frequency fluctuations.

The resulting noise model has four degrees of freedom: three amplitudes and one slope.

In addition to the noise autospectrum components, we detect the presence of a correlated noise component, i.e., a common mode between two polarization-sensitive bolometers in the same horn (e.g., 100-1a and 100-1b). The common mode was measured by analysing the noise of sum, (a + b), and difference, (a − b), time streams. If the bolometers were entirely independent, the two composite streams would have equivalent noise power. Instead, we systematically find that (a + b) has a higher noise than (a − b). The measurements for 100 GHz are presented in Fig. 25. There are indications that similar common modes exist across the focal plane, but for the purpose of noise simulation, the a/b correlated noise that directly affects polarization sensitivity is the most important one. Examples of noise model fits are presented for two bolometers in Fig. 26. Note that a common mode tracing the global 100 mK temperature fluctuations at the 1 min time scale has already been removed from the processed TOI, by reference to the dark bolometers.

5.4. End-to-end simulations

End-to-end simulations are created by feeding the TOI processing pipeline with simulated TOI to evaluate and characterize its overall transfer function and the respective contribution of each individual effect. Simulated TOIs are produced by applying the real mission scanning strategy to a realistic input sky specified by the Planck Sky Model (PSM v1.7.7, Delabrouille et al. 2013) containing a lensed CMB realization, Galactic diffuse foregrounds, and the dipole components (but without point sources). To this scanning TOI, we then add a white-noise component, representing the phonon and photon noises. The very-low-frequency drift seen in the real data is added to the TOI. The noisy sky TOI is then convolved with the appropriate bolometer transfer functions. Another white-noise component, representing the Johnson noise and readout noise with intensities derived in Fig. 26, is also added. Simulated cosmic rays using the measured glitch rates, amplitudes, and shapes are added to the TOI. This TOI is then interpolated to the electronic HFI fast-sampling frequency (40 × 181.3737 Hz). It is then converted to analogue-to-digital units (ADU) using a simulated nonlinear analogue-to-digital converter (ADC). Some 4-K cooler spectral lines are added to the TOI. Both effects (ADC and 4-K lines) are derived from the measured in-flight behaviour. The TOI finally goes through the data compression/decompression algorithm used for communications between the Planck satellite and Earth.

The simulated TOI is then processed in the same way as the real mission data for cleaning and systematic error removal, calibration, destriping, mapmaking, and power spectrum computation. The systematic effects added to the signal are simulated, with the same parameters used by the TOI processing pipeline, but no pointing error is included (unlike the simulations of planet crossings used for beam and focal plane reconstructions).

The end-to-end simulations have been used primarily to characterize our understanding of the properties of the noise in the final maps, such as the level and correlations in the noise due to known systematic effects. Figure 27 compares the PSD of noise generated by the end-to-end simulation, showing good agreement over a wide band: residuals from undetected glitches; long time-scale thermal drifts; and filtering of detector and electronics readout noise. Paper B further compares the noise properties of the end-to-end simulation maps with those of the real data.

In future releases, the end-to-end simulation will be used to characterize the filtering of the sky signal due to the analysis pipeline as a function of angular scale.

6. Conclusions

This paper has described the major improvements in the HFI time-domain processing and beam analysis in the new 2015 Planck data release. Once the ADC effect is taken into account, the bolometer gain stability is at the level of one part in a thousand and the noise is stable within 1% during the whole mission (29 months of sky survey). The ⁴He-JT 4-K cooler lines have been reduced to an insignificant level at the cost of removing 4% of the data. The beams are reconstructed out to 100′ radius at a precision that gives better than 0.1% errors in the effective beam window function. Noise and known systematic errors in the beam measurement are negligible in the analysis of the CMB data. The characteristics that can be best appreciated at the map level are described in Paper B. Overall, the instrument CMB sensitivity is 13.3 μK_CMB s^{1 / 2}. Systematic effects have been studied individually via dedicated pipelines and globally via end-to-end simulations. The main sources of systematic effects are: ADC corrections; glitch impacts; 100-mK base-plate temperature fluctuations, 4-K cooler lines, and the bolometer temporal transfer function. Their impacts have been measured in terms of transfer functions, noise, and scanning beams. The systematic errors here are considered mainly in the context of angular scales with ℓ > 10; as shown in Paper B, systematic errors remain in the data and particular affect the largest scales. A more complete description of the instrument and systematic errors in the data on the largest scales, ℓ < 10, will be described in a forthcoming publication.

Even with these significant changes in the processing of the data and the understanding of the instrument, there is room for further improvement. The ADC correction can be improved with better modelling of the raw signal and an updated model of the ADC code errors. The time response reconstruction method which was very effective here for 353 GHz can also be applied to 100–217 GHz. The cooler lines can be corrected to greater precision, allowing more samples of the TOI to be retained for mapmaking. All of these corrections in the low-level processing will yield further improvements in the polarization measurements from HFI.

---

Acknowledgments

The Planck Collaboration acknowledges the support of: ESA; CNES and CNRS/INSU-IN2P3-INP (France); ASI, CNR, and INAF (Italy); NASA and DoE (USA); STFC and UKSA (UK); CSIC, MINECO, JA, and RES (Spain); Tekes, AoF, and CSC (Finland); DLR and MPG (Germany); CSA (Canada); DTU Space (Denmark); SER/SSO (Switzerland); RCN (Norway); SFI (Ireland); FCT/MCTES (Portugal); ERC and PRACE (EU). A description of the Planck Collaboration and a list of its members, indicating which technical or scientific activities they have been involved in, can be found at http://www.cosmos.esa.int/web/planck/planck-collaboration.

References

Appendix A: HFI TOI and beam product description

Here we summarize the HFI-specific TOI and beam productes that are part of the Planck 2015 data release. A complete description is given in Planck Collaboration (2015).

All the data products are delivered as FITS files containing one or more binary table extensions. Non-data products, such as codes, are delivered as tar files.

Appendix A.1: The instrument model

Formally the “Reduced Instrument Model” or RIMO (since it is a subset of the full model maintained internally by the DPC) this file contains instrument parameters and some other product-related information.

• Detector parameters: central frequency; beam size, shape,position, and orientation in the focal plane; typical noise level;calibration factor; and polarization parameters for the PSBs.

• Map-level parameters (for the full-mission, full-channel maps only): effective frequency; beam characteristics; and noise level.

• Detector and compound spectral response profiles, for each bolometer, for each channel, and for each detector set (“Detset”).

• Detector noise spectra.

• Beam window functions and the first five error eigenmodes for the full channel maps and their cross-products, and for the Detset and single (unpolarized) bolometer maps and their cross-products.

• Beam correlation matrices, for the beams listed above.

Note that only the 50 valid bolometers are included.

Appendix A.2: TOI products

Clean and calibrated signal TOI is provided in six files, one per frequency channel, for each operational day. Each file contains the on-board time (OBT) and associated “global” flags (unstable pointing, dark correlation, first and second half-ring), and the calibrated signal of all bolometers of that channel (in K_CMB for the 100 to 353 GHz channels and MJy sr^-1 for the 545 and 857 GHz channels) each with a “local” flag (not-valid data, glitch, galactic plane, strong source, etc.).

The term “clean” means that all known systematic effects, including the solar and the orbital dipoles, have been removed, and the data, if destriped and projected onto a map using the data processing centre (DPC) tools, would yield the DPC maps. The flagged regions of the TOI (glitches, big planets, decorrelated dark bolometers) have been replaced by an estimate of the signal determined by reading the maps at the pointing position, thus they are easily recognized as regions of lower noise. This was done to preserve the continuity of the flagged timelines, a necessary step for processing them with Fourier techniques.

In addition to the signal TOI, a pointing TOI is also delivered. It has the same basic structure as the signal TOI, i.e., one file per frequency channel per operational day, but there are three columns per detector, corresponding to Galactic coordinates φ, θ, ψ for each sample, i.e., the longitude, the colatitude (= latitude + 90°), and the roll angle.

The total volume of these data is around 30 TB, of which 90% is the pointing data.

Appendix A.3: ROI products

Ring-ordered information (ROI) products contain a unique data point for each stable pointing period, or ring. In order to reproduce the maps delivered by the HFI, some baseline or destriping offsets must be applied. These offsets are computed for each detector and for each ring using the full mission data, as described in Paper B. Three sets of offsets are provided: one for the full rings, and two for each half-ring. These offsets are given in a large table which is N_rings rows × N_dets columns = 26766 × N_b, in which each cell contains a vector with the three offsets, where N_b is the number of bolometers with a delivered TOI. These values are given in the second extension. The first extension contains the indices of the first sample of each ring (and also the ESA pointingID and the ring start time) to indicate where each offset value has to be applied.

Appendix A.4: Software

• Colour-correction/unit-conversion code: this is a suite of IDLscripts to manipulate the bandpass profiles contained in theRIMO to determine colour corrections and unit conversions.

• Likelihood code: see Planck Collaboration XI (2016).

Appendix B: Scanning beam pipeline

This section describes the special processing applied to planet data, leading to the scanning beam maps, emphasizing changes from Planck Collaboration VII (2014). We use four observations of Saturn and four to five observations of Jupiter for the 100−353 GHz channels⁵. For these channels, we chose to trade the excellent cross-scan coverage of the first Mars observation for a higher signal-to-noise ratio. This choice introduced complications into the pipeline in order to handle data from observations separated in time by months, as well as vastly different planet brightnesses.

The submillimetre channels (545 GHz and 857 GHz) are driven nonlinear by Jupiter and Saturn, so we use two observations of Mars to build the beam near the peak of the main beam, supplemented by Saturn and Jupiter data to measure the near sidelobes.

The representation of the main beam’s peak and its closest sidelobes (to approximately 30′) uses B-spline functions (Planck Collaboration VII 2014), while the diffraction pattern beyond this radius follows an analytic function Aθ^-3, where A is an amplitude factor derived from the mirror edge taper (Tauber et al. 2010) and θ is the angular distance from the main beam axis. For beams using Saturn data in the peak, we apply the correction for the planet’s finite size at the effective beam window function level (Sect. 4.2). Since the scanning beam maps are evaluated in the detector frame, the scanning direction is always the same, and because the spacecraft rotation rate is nearly constant, temporal residuals can be spatially constrained in those maps. We apply a specific treatment for the time-response residual region, to improve sensitivity to the residual; specifically, B-splines are used instead of the diffraction model. See Sect. 3.4.1 for estimates of transfer-function residuals that cannot be accounted for in the scanning beam, due to the finite size of the reconstructed maps used for window function estimation.

Appendix B.1: Planet data processing

Planet data from the standard TOI processing pipeline cannot be used to recover the scanning beam to sufficient accuracy (< 1% at the window function level). We apply a suite of post-processing operations to the planet data in order to reduce the contribution of identified systematic errors to the bias in the scanning beam representation and the window function estimate. Simulations show that the additional processing applied to planetary data reduces the effective window function bias to less than the measurement errors. The following operations are applied to the planet data and are specific to the beam reconstruction pipeline:

• background subtraction;

• planet proper motion pointing correction;

• destriping;

• pointing offset correction per planet crossing;

• merging of different calibrators (renormalization);

• additional glitch residuals flagging.

The background subtraction uses the full-mission, full-frequency science maps (the planet crossings are masked). A cubic-spline interpolation is performed from the pixel centres to the planet crossing pointing (Fig. B.1).

The proper motion of the planets is corrected using ephemerides from the JPL Horizons package⁶ (Giorgini et al. 1996) to rotate the pointing into the Planck spacecraft rest frame.

The destriping removes long-timescale baseline drifts. Here we subtract a baseline offset for each circle (60 s timescales) rather than for each ring (30–60 min time scales) as in the mapmaking. We estimate the baseline offset in a range from 3° to 6° away from the beam peak as the median of the samples prior to the planet crossing, to minimize the bias from time-response residuals. We then smooth the overall trend per circle using a sliding window with a 40-circle width, as seen in Fig. B.2. Figure B.3 shows an example of the destriping procedure.

During the destriping process, we flag and remove the entire circle if one of the following criteria is met in the baseline timeline:

• the number of flagged samples exceeds 25% of the total samples inthe circle;

• there is an event exceeding 3.3 times the estimated local rms (residual glitch biasing the estimate of the baseline);

• the number of positive and negative samples is not balanced, such that their relative contribution in the total number of used samples differs by more than 20%.

An additional 5% of the planet data are flagged during this procedure, largely in the main-beam area.

We expect an error of a few arc seconds in the pointing centroid of each planetary observation, given the current pointing model (Planck Collaboration I 2016). We apply a re-centring procedure to each planet crossing by fitting a beam template to each observation, where the free parameters include the pointing shifts in both directions (co-scan and cross-scan). The pointing of each planet crossing is corrected with these offsets. The planet data processing is an iterative process, with a loop over the beam template used. For the first iteration, the template is a prior version of the beam.

Each planet crossing has a different peak brightness, so we renormalize the planet signals in order to merge them into a common beam map. We interpolate the beam template to the pointing of each crossing to take into account the non-uniform coverage of the calibrator observation and perform an azimuth angle average (see Fig. B.4) of the samples. We then recover an amplitude factor by fitting the azimuthally-averaged planet data to an azimuthally averaged interpolated template. For Jupiter data, only the portion of the profile not affected by nonlinearity is used in the fitting procedure. Since the same template is common to all calibrators, this procedure, by construction, forces agreement between the planet data. Figure B.5 shows the renormalized profiles. We apply an interpolation on the two-dimensional beam map template to take into account the change in the coverage per scan, distorting the uniformly distributed azimuthal template, since beams are asymmetric and the sampling is sparse during a planet observation due to the Planck scan. The spread at low radius between different scans and planets is dominated by this effect, while the spread at high radius is mainly attributed to the different signal-to-noise properties of each calibrator.

The performance of the standard TOI deglitching is degraded by the high signal gradients of planet scans, relative to CMB data; with no additional flagging, glitch residuals create a bias of several percent in the effective beam window function. We therefore add an extra stage of glitch flagging to the planet data processing that excludes samples exceeding 3 times the estimated local noise rms in the low gradients region of the main beam, and 3 times the expected rms of high-frequency pointing errors in the strong gradient regions of the main beam (see Fig. B.6). The fact that the excluded samples are vertically aligned when projected on the cross-scan axis indicates that they are likely to be within the same scanning circle, correlating them in time. The co- and cross-scan projection indicates they are localized in the beam peak region, where the performance of the standard glitch removal process is seriously affected.

The planetary data processing converges in five iterations to within the Monte Carlo error bars (see Fig. B.7), and is not dependent on the template used in the first iteration.

Appendix B.2: Hybrid representation for main beams

We use Saturn data to measure the main beam and Jupiter to measure the closest sidelobes for all channels, except 545 and 857 GHz where the peak is described with Mars data. The transition regions consists of a linear weighting scheme between the data sets. The transition region is defined as the span of signal samples that are lower than 9 times the estimated noise rms of the peak planet (Saturn or Mars) and greater than the Jupiter threshold. The Jupiter threshold is set such that we keep only data where nonlinearities of the detectors and the finite size of Jupiter’s disc have negligible effects compared to the noise uncertainties. Table B.1 summarizes the Jupiter threshold levels in peak maximum amplitude. They have been set to conservative values.

We further spatially average the combined Jupiter data in regions where the signal is lower than 3 times the estimated Jupiter rms noise. Physical optics simulations show that we should not expect high-spatial-frequency variations in the beam in those regions. The regularization filter is a square boxcar average, 0.́5 on a side for 100–353 GHz, and 10′′ on a side for 545–857 GHz.

For the final beam model, we fit the B-spline coefficients to the renormalized planet timelines and the filtered Jupiter data, using a least squares estimator. As opposed to Planck Collaboration VII (2014, in particular, their Eq. (C.4)), no additional smoothing criterion is added to the scoring function in the inversion process, since the signal-to-noise ratio of four Saturn crossings is high enough to prevent bias in the B-spline functions coming from high-frequency noise. The inversion process uses the Cholesky method from the LAPACK module (Anderson et al. 1999). Table B.2 summarizes the control point (knot) parameters. The knot locations are tuned so that the final bias at the window function level is included in the error budget. The geometrical shape of the B-spline functions defines the properties of the intrinsic low-pass filter acting on the data, while preventing the reconstruction being too sensitive to high frequency errors. The parameter choice relies on the data quality and the coverage of the combined planet scan.

In the transition area describing the near sidelobes, we further process the B-spline representation in order to capture the spatial variations of this intermediate regime while allowing a smooth transition into the analytic diffraction pattern. To do this, we build azimuthally-averaged profiles in a number of ranges of azimuth angle, and propagate these templates with a spline interpolation to the final map. The number of azimuthal templates is constrained by the expected asymmetries and secondary lobes derived from optical predictions in this regime of the main beam. Table B.3 summarizes the number of angular regions at each frequency band. Since the azimuthal templates are equally distributed in angle, the number of templates sets the separation between each angularly averaged template, and hence the fidelity of the reconstruction to the azimuthal asymmetries of the main beam in this regime.

Figure B.8 shows the azimuthally-averaged profiles as a function of the radius from the beam centroid. The differences in the templates beyond a 5′ radius show the optical asymmetry of the beam. In addition, the templates close to 90° azimuth angle, shown in blue, show time response residual effects. In this region, the initial B-spline evaluations are kept with no additional azimuthal averaging, in order to better reproduce the uncertainties in the time response of the detector and readout electronics. The templates are only used in the regions where Jupiter’s rms noise is not negligible, down to the assumed symmetric diffraction pattern dominant regime.

The intermediate sidelobes beyond 30′ from the beam centroid are dominated by a diffraction pattern and are modelled from optical predictions. The predictions are compatible with Jupiter azimuthally-averaged profiles (see, e.g., Planck Collaboration XXXI 2014). We make the assumption that the intermediate sidelobe behaviour is azimuthally symmetric and we keep the B-spline representation in the transfer function residual region (temporal causal part of the beam) to properly account for temporal transfer function residuals.

Appendix C: Time response

Appendix C.1: Measurement of 353 GHz bolometer time constants

As for the other HFI detectors, the 353 GHz bolometer thermal time response is described in the Fourier domain as the sum of single-pole low-pass filters, each parameterized by a time constant and an amplitude. However, the measurement procedure followed a different process than for the 100–217 GHz bolometers (described in Sect. 3.4), jointly solving for the optical beam and time response. When applied to 353 GHz bolometers, the nominal time response procedure created scanning beams with strong (several percent of the peak amplitude) non-optical asymmetries in the scanning (also causal in time) direction, extending beyond 30′ from the beam centroid. This indicated that the recovered time constants from the previous method were far from the true ones, distorting the corresponding scanning beams. Therefore, a new method was developed.

The procedure described here, created for the 353 GHz time response, will potentially be used in the future at all frequencies, due to the improved recovery of the very low frequency portion of the time response, as well as better reconstruction of the scanning beam close to the centroid.

Appendix C.1.1: Normalization of the glitch slow component

The stacked glitch template used for glitch removal (Sect. 3.2 and Fig. 10; see also Planck Collaboration X 2014) is built using data filtered with a 3-point finite impulse response filter⁷. Hence, the time response inferred from glitches, and in particular the relative amplitude of slow response to fast response, is strongly biased and must be renormalized.

Appendix C.1.2: Starting parameters and dipole correction

The time constants and associated amplitudes where τ> 300 ms are set to the glitch values, but renormalized in overall amplitude using planet data with the scanning beam method described in Appendix B. The results of the fitting procedure for the time response is dependent on the initial guess because of the strong correlation in parameters and because planet data (considered only within 100′ of the centroid) only weakly constrain the very slow portion of the time response (τ> 1 s). The initial guess of the glitch renormalization amplitude is set such that the created power compared to the previous release matches the correction needed to remove half of the shifted dipole effect (Planck Collaboration VIII 2016), assuming that the other half of the change will be provided by the change of the fast components of the time response.

Because of the convention we adopted for the global normalization of the time response model, any additional response at very low frequencies added to match the long timescale response of the glitches changes the amplitudes of the existing time response. Thus, for all the other parameters, the starting values are the previous-release parameters, with amplitudes adjusted to include the power introduced by the renormalized glitch template in the time response.

Appendix C.1.3: Beam-forming while recovering the parameters

The simultaneous fitting of an optical beam shape and time-response parameters for the 353 GHz detectors follows the procedure outlined in Planck Collaboration VII (2014), apart from slight differences in the distance from the planet centroid where the fitting is performed. A symmetrized beam profile along the scanning direction is derived from initial-guess time response parameters, then updated after each iteration.

The fit is performed as a loop over two steps, alternating fast (10 ms <τ< 100 ms) and slow (100 ms <τ< 300 ms) parameter adjustments. For fast parameters, the recovery is performed over the range 3′ to 70′ from the beam peak centroid, while for slow parameters the range starts at 20′ and extends up to 300′. In the first step, the fast parameters are fitted to combined Saturn and Jupiter data, with other parameters fixed (intermediate ones and glitch template amplitudes). The second step adjusts intermediate parameters and the glitch template amplitude to match Jupiter data with other parameters fixed (fast parameters). We stop the loop when both of the following requirements are met: (1) the symmetrized beam template is stable within 1% accuracy in the peak domain; and (2) the relative reduced χ² is stable within 10^-4 in each iteration. Figure C.1 illustrates the overall process for the 353 GHz channels. The final parameters are listed in Table C.1. Notably, this solution requires the inclusion of two extra time-constants and associated amplitudes.

All Tables

All Figures