© ESO 2020

1. Introduction

Since its launch in December 1999, ESA’s space-based X-ray telescope XMM-Newton (Jansen et al. 2001) has observed about 1150 square degrees of the sky in pointed and mosaic mode, and about a third of that area several times. From all public X-ray data taken with its EPIC¹ CCD instruments MOS1, MOS2 (Turner et al. 2001), and pn (Strüder et al. 2001), the XMM-Newton Survey Science Centre Consortium (SSC, Watson et al. 2001) compiles the serendipitous source catalogues. Their third generation named 3XMM (Rosen et al. 2016) was augmented by the first source catalogue from overlapping observations, 3XMM-DR7s (Traulsen et al. 2019). Other major source catalogues based on XMM-Newton observations include the XMM-Newton Slew Survey Source Catalogue (Saxton et al. 2008) of X-ray sources observed by EPIC pn during telescope slews with its latest edition XMMSL2² and the OM Serendipitous Ultraviolet Source Survey Catalogue (Page et al. 2012) of sources observed by the Optical Monitor (Mason et al. 2001) with its latest edition SUSS4.1³.

After upgrades to the catalogue pipelines and the underlying SAS⁴ tasks, all XMM-Newton EPIC data were fully re-processed⁵ by the XMM-Newton Science Operations Centre (SOC) at the European Space Astronomy Centre (ESAC). Based on these data, the SSC compiled the fourth generation of serendipitous source catalogues. The new 4XMM-DR9 catalogue from all individual observations is described by Webb et al. (2020) and the second catalogue from overlapping observations, named 4XMM-DR9s, is explored in this work. To produce 4XMM-DR9s, observations that overlap by at least 1′ in radius are grouped into stacks and processed together. Source detection is performed simultaneously in all images of all observations in a stack. The number of input images for a detection run is thus given by the total number of exposures⁶ for this stack times the number of energy bands (which is five in the serendipitous source catalogues).

This paper on the second catalogue from overlapping XMM-Newton observations describes the updates with respect to Traulsen et al. (2019) to the underlying software and the respective processing parameters in Sect. 2. In particular, an astrometric field rectification prior to stacked source detection is new to this catalogue and introduced in Sect. 2.3. Updates to the field selection and the catalogue processing are described in Sect. 3. Section 4 is dedicated to the structure of 4XMM-DR9s, the revised and newly introduced columns, and the cross-match with 4XMM-DR9. Positional accuracy, source content and information in comparison to 4XMM-DR9, and long-term variability of catalogue sources are studied in Sect. 5. Section 6 summarises the results.

2. Updates to data processing and source detection

The data processing for 4XMM-DR9s was based on the event lists and attitude files produced for 4XMM-DR9 and essentially followed the strategy of Traulsen et al. (2019). Mosaic-mode observations, which consist of a series of exposures, were split into several observation identifiers by the pipeline, creating one for each sub-pointing (cf. Rosen et al. 2016). They were thus treated like independently overlapping observations by stacked source detection. The data of each stack were projected tangentially onto the same reference coordinates with respect to the centre of the full area covered by all observations in the stack. The centre was calculated from the attitude coordinates of the contributing observations as the mean of minimum and maximum right ascension and declination. Input products for source detection were created for each observation, consisting of images, background maps, exposure maps, and detection masks.

The source-detection method was described in detail by Traulsen et al. (2019). Images, exposure and background maps were created for each exposure and energy band, and detection masks were created for each exposure. They were used in the two-step source detection process, which was simultaneously run on all images in the stack. Sliding-box source detection was performed to create an input list of tentative source positions for the main step, a maximum-likelihood PSF fit in all images of the contributing observations, instruments, and energy bands. Using the three EPIC CCD instruments pn, MOS1, MOS2, and the five XMM-Newton standard energy bands⁷, the fit thus involved five, ten, or fifteen images for each observation. The point-spread functions (PSFs) at the tentative source position and mid-band energy were read (cf. sketch in Fig. 1), taking the different PSF shapes and distortions in the individual contributing images into account. The 2d model of the point-spread functions was derived by Read et al. (2011) and involves a Gaussian core, a King component, and the primary and secondary spokes that arise from the mirror structure and scattering. In the simultaneous maximum-likelihood PSF fit, the count rate in each image and a common source position and source extent were determined. Position and extent were considered to be the same in all images since they are not expected to vary between observations, while the count rate and all derived parameters were determined in each image individually. A detection likelihood was calculated from the count rate under the PSF using Cash statistics (Cash 1979) and the null hypothesis that the measured signal arises from pure background fluctuations. Source extent was tested by convolving the point-like PSF with a beta model, whose radius was a free parameter of the fit. A minimum extent radius of 6″ and a log-likelihood difference of at least four between the fits with the extended and the point-like PSF were required to accept the extended fit. Otherwise, the source was considered point-like. All source parameters, in particular the detection likelihood, were derived from the combined fit. The fit results of all images were used to calculate the all-stack source parameters. Observation-level parameters were calculated from the same fit results by using the subset of the images of each observation separately (Fig. 1). Comprehensive parameter lists are given in the Appendix of Traulsen et al. (2019) and of this paper. Its structure is also shown in Fig. A.4. The catalogue includes sources which have a detection likelihood of at least six in the total stack or in a contributing observation. The likelihood threshold of six is used in all XMM-Newton serendipitous source catalogues.

Fig. 1. Illustration of the PSF shapes in the three EPIC instruments at different off-axis and azimuth angles in overlapping observations and of the parameter determination in the simultaneous maximum-likelihood PSF fit. The normalised PSFs – shown here for 1 keV in a linear intensity scale – are scaled to the count rate in the respective image within the detection fit. Position and extent of a source are the same in all observations, while the count rates are fitted in each image. Observation-level and stacked parameters are derived from the parameters of the contributing images (Traulsen et al. 2019).

2.1. Task updates

Several source-detection SAS tasks have been revised since publication of the first catalogue 3XMM-DR7s, in particular esplinemap⁸ producing the background maps, eboxdetect for the first sliding-box detection step, emldetect performing the maximum-likelihood source detection, and srcmatch, which was used to create merged 4XMM-DR9s input source lists and the final 4XMM-DR9 source lists. The most relevant changes are described in this Section. More details on all updates are included in the task history of the SAS packages, which is for example part of the SAS release notes⁹.

For all 4XMM catalogues (DR9 and DR9s), a pile-up estimate has been introduced (Webb et al. 2020). It is calculated for each detection and EPIC instrument within emldetect. Also, emldetect gives the extent likelihood now for all sources, including those fitted with a point-like PSF (cf. Sect. 4.4 and Webb et al. 2020).

To handle large stacks, runtime and memory requirements of several tasks were reduced, in particular for esplinemap and emldetect. Runtime is crucial for very deep stacks with many directly overlapping observations. Memory is crucial for very extended stacks containing observations that cover several square degrees on the sky in total. Depending on the depth and size of the stack, emldetect for example consumes up to 30% less memory after the revision and becomes faster by 20% to 90%. These enhancements and new functionalities become public with the release of SAS 19.

In addition, SSC-internal versions of eboxdetect, emldetect, and srcmatch were established which can process large stacks of up to 400 observations. They are automatically chosen by the catalogue pipeline whenever necessary. These updates and the pipeline are not part of the public SAS releases, since compiler options deviating from SAS standards are used to produce them and hardware beyond standard PCs is required to run them.

2.2. Background modelling: new default parameters for all 4XMM catalogues

To determine the background at the tentative source positions during source detection, a background map was created for each input image and the modelled signal of out-of-time events added to the EPIC-pn maps¹⁰. The background structure was modelled based on the source-free image regions. For catalogues prior to 3XMM-DR7s, it was approximated with a spline fit. Traulsen et al. (2019) employed an adaptive smoothing technique for the first catalogue from overlapping observations, which was created from good-quality data. The exposure-normalised images were convolved with a Gaussian kernel and divided by the detection masks which were smoothed using the same kernel. For increasing kernel widths, the signal-to-noise ratio in each pixel was derived from the counts under the kernel. The background value for which the intended signal-to-noise was reached was written to the background map. Parameters of the adaptive smoothing method are the signal-to-noise value, the minimum kernel width, and the brightness threshold above which sources are excised from the input image. This approach has now been used for all 4XMM catalogues consistently. To work with single and stacked observations of very different background levels, the smoothing parameters were re-investigated as described in Webb et al. (2020). A brightness threshold for the source cut-out radius of 2 × 10⁻⁴ counts arcsec⁻², a minimum smoothing radius of 10 pixels (40″ in default image binning), and a signal-to-noise ratio of 12 were established for 4XMM-DR9 and for 4XMM-DR9s.

2.3. Event-based astrometric correction

As described in Rosen et al. (2016), celestial coordinates of sources emerging from the PSF fitting step of a given observation include a generally small systematic error arising from offsets in the spacecraft boresight position from the nominal pointing direction for the observation. Such systematics were removed from source lists of individual observations by applying the SAS task catcorr which reveals shifts in RA and DEC and a field rotation angle via probabilistic matching of X-ray sources with sources listed in SDSS (Ahn et al. 2012), USNO-B1.0 (Monet et al. 2003), or 2MASS (Skrutskie et al. 2006). The procedure for 4XMM-DR9 is described in Webb et al. (2020).

When several overlapping observations are to be processed, such corrections can only be applied before simultaneous source detection and not at the end of the processing, because individual OBS_IDs have their own catcorr corrections. We therefore introduced an event-based astrometric field rectification during pre-processing, shifting the recorded events to corrected positions. The catcorr results per observation, derived within the 4XMM-DR9 processing, were directly applied to the relevant header keywords of the attitude files and to the attitude coordinates, which were later used to project the event lists onto the common coordinate system of the stack. Stacked source detection was then performed on the individually modified event lists. The distribution of the shifts applied to the 4XMM-DR9s input observations is shown in Fig. 2. The plots include the results from 2516 fields that were corrected against SDSS positions, 2267 fields against USNO-B1.0 positions, and 842 against 2MASS positions. The astrometric accuracy of all three comparison catalogues is of the order of 0.1″-0.2″. For 4XMM-DR9 observations that could not be corrected, a mean systematic astrometric uncertainty of about 1.3″ remains (Webb et al. 2020).

Fig. 2. catcorr results for the input fields to 4XMM-DR9s which could be astrometrically corrected: histograms of the applied shifts along right ascension (left, red crosses, ΔRA × cos(Dec)) and declination (left, blue pluses, ΔDec), the additional field rotation (right), and the resulting absolute field shift (left, black).

Before compiling the full 4XMM-DR9s, we verified the approach on a test sample of 25 stacks with large catcorr shifts. They comprised two to six observations with total offsets of up to 3.8″. Figure 3 shows the differences between X-ray and optical SDSS positions in the test sample with and without positional rectification (matching radius 3″). The success of the chosen approach becomes rather obvious (on the remaining deviations from an ideal Rayleigh distribution, cf. Sect. 5.1). The number of matches and the positional accuracy of the X-ray sources from stacked source detection both increase. The improvement in positional accuracy was independently confirmed through a match with Gaia-DR2 (Gaia Collaboration 2018) which was not used by catcorr, again within 3″.

Fig. 3. Distribution of the angular distances, given in units of the statistical error, between X-ray and optical SDSS positions before (blue) and after (red) event-based field rectification in a test sample of 25 stacks. The grey curve shows an ideal Rayleigh distribution.

3. Field selection and catalogue processing

3.1. Determination of the background cut

Exposures affected by high background emission may reduce the detection reliability and the accuracy of the source parameters in a stack. In extreme cases, the number of tentative input sources to the maximum-likelihood fitting, which are selected by a sliding-box detection algorithm, may even become too large to process the stack at all. As for 3XMM-DR7s, we thus defined background cuts above which an exposure is discarded from stacked source detection. They were determined following the general method outlined in Traulsen et al. (2019), now derived individually for each camera, each filter, and each observation mode. This method was employed to select or discard exposures from the catalogue sample. The background maps used in source detection are described in Sect. 2.2.

In the source-excised full-band images of all 4XMM-DR9 exposures, we determined the mean background count rate per unit area. For EPIC pn, the rate was calculated for each quadrant separately, where one quadrant comprises three CCDs. For EPIC MOS, it was calculated for each CCD separately except for the central one. Usually, the target of the observation is placed on this central CCD, and if a large area had to be cut out around a bright or extended target, the remaining area might be too small for a reliable background determination. Correspondingly, the other CCDs were considered in the analysis if at least 100 valid pixels remained after masking the sources. The maximum rate among all usable pn quadrants or MOS CCDs was chosen in each exposure and the distribution of background rates derived for each filter-mode combination. While the results for the three imaging modes, which were used in the catalogues, were similar within their uncertainties, clear discrepancies were found between imaging mode and timing mode and between exposures obtained with different filters (Fig. 4). An empirically chosen Lorentz function (Traulsen et al. 2019) was fitted to the rate distribution of all exposures taken in the same imaging mode and filter. In MOS timing mode, too few exposures were available to fit the filters individually, thus one fit was used for all of them. Since count rates are instrument-dependent and cannot be compared directly if taken in different technical setups, we opted for an independent quantity to characterise the background level. From the Lorentz functions, we thus derived the associated Cauchy cumulative probability distributions (examples shown in the lower panel of Fig. 4). The Cauchy probability attributed to an exposure then serves as a configuration-independent measure of the background level, which catalogue users can access to apply their own stricter restriction on the background level consistently without caring about technical details of the individual observations.

Fig. 4. MOS2 background levels between 0.2 and 12.0 keV for the observing modes full frame, large window, small window, and timing in thin filter (upper panel) and for the thin, medium, and thick filter in full-frame mode (lower panel). The dashed and dotted lines show the upper part of the corresponding Cauchy probability values for the different filters.

To choose the final high-background cut, we selected sky areas which were observed repeatedly at very different background levels and performed source detection in the individual observations and their stacks. The usable sources at the different background levels, the sources in the stack, and those in the contributing observation with the lowest background were compared. From the results and from additional visual inspection of the images using a brightness scale depending on the exposure time, we chose a background cut of 0.92. Exposures with a higher Cauchy probability tend to degrade the source-detection quality and were rejected. The remaining observations are considered suitable for stacked source detection.

3.2. Exposure selection

4XMM-DR9s is based on a subset of all observations that entered 4XMM-DR9. Discarded from stacked source detection were: (1) observations with too small an overlap; (2) pn exposures in small window mode; (3) the central MOS CCDs of exposures in small window mode; and (4) exposures with too high a background (see Sect. 3.1).

Observations were grouped into a stack if they overlap by at least 1′ in radius, which corresponds to the maximum radius of the fit region of a point source in the catalogue processing and ensures that the fit region is fully covered by the combined observations. Thus, observations were selected if their aim points have a maximum distance of 29′, referring to the mean field coordinates given in the headers of the 4XMM-DR9 source lists. Because of the offset between the pn and MOS detectors, the area covered by the EPIC instruments is slightly asymmetric. In terms of actually exposed area, 29 observations therefore overlap by less than 1′ with their neighbouring observations although formally meeting the 29′ criterion. These were de-selected manually. Another 88 candidate stacks which met the selection criteria (1) to (3) were discarded because the background level of all their observations or of all but one was too high.

The final selection comprises 6604 observations with at least one usable exposure in 1329 stacks. Their depth and distribution over the sky are shown in Fig. 5 in a Hammer-Aitoff representation and their observation modes, filters, and usable chip area according to their OBS_CLASS classification in 4XMM-DR9 in Fig. 6. The number of observations in the stacks are given in Fig. 7. Two thirds of the stacks comprise two or three observations. The largest stacks are the XXL North and South regions with 352 and 266 observations, of which up to eleven overlap directly at a given sky position. Figure 8 shows the depth of all stacks as cumulative repeatedly covered sky area over total exposure time. Out of about 485 square degrees total sky area in 4XMM-DR9s, 300 square degrees were multiply observed with a maximum exposure time of about 1.9 Ms.

Fig. 5. Sky map of the 4XMM-DR9s input observations in a Hammer-Aitoff projection in Galactic coordinates. The radius of the plot symbols corresponds to 2°, thus to four times the actual field-of-view of XMM-Newton. Colour codes the total exposure time of the overlapping observations in each plotted bin.

Fig. 6. Distribution of observation modes, filters, and usable chip area OBS_CLASS in 4XMM-DR9s. Possible modes are extended full frame (EF), full frame (FF), large window (LW), small window (SW), timing (TI). Filters include thin (TN), medium (ME), and thick (TK). Ten observations have the poorest OBS_CLASS 5.

Fig. 7. Number of observations in the 4XMM-DR9s stacks.

Fig. 8. Cumulative sky area of multiply observed patches over exposure time in the input sample. “All-EPIC” shows the total exposure, for which the maximum exposure among the active EPIC instruments is determined in each observation and sky pixel and then summed over all observations.

3.3. Catalogue processing

The data processing strategy, starting from the modified 4XMM-DR9 attitude files and event lists, and source detection were described in Traulsen et al. (2019) and not changed. In addition to the standard products needed to run source detection, the pipeline produces coverage maps, mosaic images, and exposure maps for each stack, which are newly published together with the catalogue (Sect. 4.5).

While the largest stack in 3XMM-DR7s comprised 66 observations, the catalogue pipeline for 4XMM-DR9s was revised to be capable of processing several hundred observations and deep stacks with dozens of directly overlapping observations. The standard tasks published in SAS18 were modified to speed up preparation time by parallel pre-processing of the observations. Using these software versions, the catalogue could be produced on standard PCs and a server with higher memory capacity. Figure 9 shows an example detection image and exposure map of a pipeline-processed stack comprising six observations.

Fig. 9. Example of a stack with six observations, targeting the star cluster NGC 2264. Upper panel: false-colour image in three energy bands 0.2 − 1.0, 1.0 − 2.0, and 2.0 − 12.0 keV to illustrate the different flux maxima of the detected objects. Lower panel: exposure map, giving the total exposure in ks, taking vignetting into account, with 4XMM-DR9s sources as red circles and 4XMM-DR9 sources as blue diamonds. The symbol size scales with the core radius of the extent model. Several low-quality (flagged) DR9 detections were not recovered in DR9s.

4. The second edition of the catalogue from overlapping observations: 4XMM-DR9s

4.1. Basic properties

The serendipitous XMM-Newton source catalogue from overlapping observations is composed from the source lists of all stacks. It consists of several rows for each individual source, which is identified through its IAU name 4XMMs Jhhmmss.s±ddmmss and a unique identifier SRCID. A summary row lists the source parameters determined for the full stack of observations. The following rows give the source parameters for each contributing observation separately. They are also derived from the same stacked maximum-likelihood fit (cf. Sect. 2). Stack-specific columns can be used to restrict the catalogue to its summary rows, for example by choosing N_CONTRIB > 0 for all sources and N_CONTRIB > 1 for sources in overlap regions. Observation-specific rows can be selected for example via the observation identifier OBS_ID. An overview of the catalogue layout including a column list and screen shots are given in Appendix A.3. All detected sources from the input observations are transferred to the catalogue, whether located in overlap areas or not. A newly introduced boolean flag OVERLAP is set to true in all rows of the sources with at least two contributing observations and thus allows for direct selection of multiply observed sources.

4XMM-DR9s comprises 288 191 unique sources from 6604 observations, and 218 283 of them were found in overlap areas. The other 69 908 sources were observed only once. These are located in the outer parts of the stacks or in smaller regions missing from one of the observations like a chip gap, the gaps in MOS large-window mode observations, or on the de-activated MOS1 CCDs. The higher source density through stacked source detection is plotted in Fig. 10 in terms of extrapolated source number per square degree over the number of contributing observations and over the cumulated exposure time, taking vignetting effects into account.

Fig. 10. Source density in 4XMM-DR9s over number of contributing observations (left panel) and total exposure time in all contributing observations, taking vignetting into account (right panel). Only small sky areas are covered by more than 30 observations or for more than 0.5 Ms (cf. Fig. 8), causing a wider scatter in the distributions.

Seven percent of the sources are detected as extended with a core radius of at least 6″, and 88% have a good or very good quality flag (STACK_FLAG ≤ 1, referred to as “un-flagged” throughout this paper, cf. Sect. 4.2). The full-band median fluxes and flux errors of the catalogue sources are 1.7 × 10⁻¹⁴ erg cm⁻² s⁻¹ and 6.1 × 10⁻¹⁵ erg cm⁻² s⁻¹ for multiply observed sources and 2.1 × 10⁻¹⁴ erg cm⁻² s⁻¹ and 9.5 × 10⁻¹⁵ erg cm⁻² s⁻¹ in non-overlapping areas, respectively. About 4% of the un-flagged multiply observed sources show signs of high long-term inter-observation variability according to VAR_PROB ≤ 10⁻⁵. VAR_PROB, one of the variability parameters which were introduced for 3XMM-DR7s (Traulsen et al. 2019), is the probability that the fitted source fluxes are consistent with constant flux in all observations (cf. Sect. 5.3). An overview of the catalogue properties and a comparison to 3XMM-DR7s are given in Table 1.

Figure 11 illustrates that through stacked source detection a larger fraction of low-flux sources is uncovered in the overlap areas (N_CONTRIB > 1) compared to sky areas observed once (N_CONTRIB = 1). The lower median flux in overlap than in non-overlap areas, given in the paragraph above, corresponds to the relative flux histogram peaking at lower fluxes (upper panel of Fig. 11). The cumulative representation (lower panel of Fig. 11) shows the higher number of detected low-flux sources in the overlap areas. 4XMM-DR9s contains 6142 sources that have low detection likelihood (EP_DET_ML < 6) in all contributing observations and become significant (EP_DET_ML ≥ 6) only through stacked source detection. Their parameters are plotted in Fig. 12: contributing observations, total detection likelihood, counts, and flux. New sources with up to 2000 counts and sources with a detection likelihood up to 50 are found that were missed when the observations would be processed individually. They are located in areas with eight to forty directly overlapping observations. Section 5.2 discusses the source gain compared to the catalogue from individual observations 4XMM-DR9.

Fig. 11. Flux distribution of multiply observed sources (blue) and of sources covered once (red) in 4XMM-DR9s. Upper panel: histograms normalised to the sample size. Lower panel: cumulative histograms.

Fig. 12. Parameters of sources that surpass the minimum detection likelihood only in the stack summary. Their observation-level detection likelihoods in the images of a specific observation, which are derived from the same simultaneous fit, are all below the limit of six.

4.2. Quality assessment and visual screening

All XMM-Newton catalogues include information on the quality of a detection in terms of flags that indicate whether it is potentially spurious or located in a problematic region that could affect the reliability of its parameters. The initial quality assessment is performed automatically by the SSC-internal SAS tasks dpssflag within the processing pipeline, following the strategy of Watson et al. (2009). It defines nine boolean flags for each exposure of a source, coding several poor observing conditions which potentially deteriorate the accuracy of the detection or its parameters. The flags enter the catalogue as nine-character strings ii_FLAG, where ii stands for one of the instrument abbreviations PN, M1, M2. An overview of their definition is given in Table 2. The all-EPIC string flag EP_FLAG is true wherever an instrument flag is true. The integer STACK_FLAG summarises the nine boolean flags and is “0” in case of no warnings, “1” for reduced detection quality in at least one image, “2” for potentially spurious sources (cf. Table 2), and “3” in the summary row for flags “2” during all contributing observations. The cleanest set can thus be chosen by applying the filtering expression “STACK_FLAG ≤ 1”, while detections with higher flags can still come from real sources, but may have uncertain fit parameters. Subsequent quality assessment is done through visual inspection of all source-detection results. The first catalogue from overlapping observations 3XMM-DR7s was based on a clean sample and used the automatically set flags only. 4XMM-DR9s includes significantly more observations taken under various and partly unfavourable conditions. Therefore, additional manual flags are introduced with this version, which resemble those in the series of catalogues from single observations.

The results of source detection in each stack were screened visually making use of SAOImage DS9 with XPA (Joye & Mandel 2000) and STILTS (Taylor 2005) commands. Obviously spurious sources are flagged manually, (a) if detections lie on single reflection patterns, (b) if several detections are found on detector features like clusters of bad pixels without any sign of a blended real source, (c) if they are heavily confused with bright extended emission and thus have unreliable source parameters, (d) if several detections lie directly on the PSF spikes of bright sources, (e) if a bright source is fitted with several off-set detections because of extreme pile-up, (f) if emission of a very bright target triggers spurious extended detections, or (g) if multiple overlapping sources are fitted to large extended emission or to the footprint of a Solar System body. The screening process is not meant to be complete, but shall help users to reject evidently not genuine sources. Ambiguous cases remain un-flagged in the visual screening to reduce the risk of erroneously excluding good detections, since they are prone to visual mis-classification, for example whenever a real source might be overlapping with a detector feature, if few non-overlapping extended detections cover large extended emission, or if two detections are used to describe the emission of probably only one real source. Sources can have automatically set flags and no manual flag, for example in the vicinity of bright or extended sources where their source parameters could be affected by higher uncertainties than in clean areas. Figure 13 shows examples from the screening process.

Fig. 13. Examples from the manual screening process: bad detector features (panel a), single-reflection patterns (panel b), RGA diffraction pattern (panel c), path of a planet in a mosaic-mode observation (panel d). Yellow circles show all detections, where large radii correspond to the core radius of extended sources. Bad regions or detections are marked manually (green). The detections within them (red) are then flagged in the catalogue.

Manual flags are indicated in the STACK_FLAG and EP_FLAG columns: EP_FLAG is expanded from nine to ten characters by an additional flag (character) at the end of the string which is true (“T”) for manually flagged sources and false (“F”) for the rest. STACK_FLAG is increased by ten if a manual flag was set. The results of the automated flagging are thus preserved in its units’ digit, while the manual flag appears in its tens’ digit. A filtering expression “STACK_FLAG ≤ 1” removes both sources with the highest automatic flags 2 and 3 and sources with a manual flag. Eventually, 22 892 catalogue sources received a manual flag and additional 8 495 have STACK_FLAG 2 or 3.

4.3. Cross-matching with the 4XMM-DR9 catalogue

The catalogue 4XMM-DR9 from individual observations (Webb et al. 2020) is distributed in two flavours: from the source lists of all individual observations, the catalogue of detections is created. Sources can thus be included several times in the 4XMM-DR9 catalogue of detections, if they were observed repeatedly. By a positional match, individual detections from different observations are merged into unique sources, and the parameters of the individual detections are merged into unique source parameters. In particular, an averaged unique source position and position error are calculated from the positions and errors of the individual detections. The results of the merging compose the 4XMM-DR9 catalogue of unique sources, also referred to as “slim version” in other publications.

The catalogue 4XMM-DR9s from overlapping observations is matched with a subset of 4XMM-DR9 beginning with the catalogue of sources. The 473 488 unique DR9 sources which have at least one contributing detection with a good summary flag, SUM_FLAG ≤ 1, are selected. The search radius is set to 2.2698 times their statistical and systematic position errors, corresponding to the 99.73% confidence region of a Rayleigh distribution. In DR9, the combination of the errors is given in the column $\mathrm{SC}\_\mathrm{POSERR}/\sqrt{2}$. In DR9s, we use the statistical position error given in the column $\mathrm{RADEC}\_\mathrm{ERR}/\sqrt{2}$ and the additional error component derived from a fit to a Rayleigh distribution in Sect. 5.1.

For each 4XMM-DR9s source, the nearest associated DR9 source is included in the 4XMM-DR9s catalogue. In particular, if more than one DR9 source is found within the matching radius, the one with the closest position is chosen. The DR9 identifier, position, quality and variability flags are copied from the 4XMM-DR9 catalogue of sources into the stack-level rows (for the structure of 4XMM-DR9s, cf. Fig. A.4). From the DR9 catalogue of detections, the parameters are copied into the DR9s observation-level rows of the associated source. The distances between the DR9s and the DR9 positions are calculated and also listed. Columns with DR9 information are marked by the suffix _4XMMDR9. If no association is found, the table cells remain empty. The matching results are further discussed in Sect. 5.2.

4.4. New and revised columns in 4XMM-DR9s

With 4XMM-DR9s, 26 columns are newly introduced with respect to 3XMM-DR7s. The definitions of three more columns were revised, and all columns from the match with the catalogues from individual observations were renamed. These changes are listed in this Section. An overview of all 4XMM-DR9s columns is given in Table A.1 and more details on all the columns that were initially defined for 3XMM-DR7s in Traulsen et al. (2019).

N_EXP. Analogously to the number of contributing observations N_CONTRIB, N_EXP gives the number of exposures used in the fitting process, which is the sum of active instruments in all contributing observations. If all instruments were active during all observations, it is thus 3 × N_CONTRIB in the summary rows. Exposures may be missing because an instrument was operated in timing mode for example, because of technical problems, or because they were excluded from the catalogue processing due to a high background level (Sect. 3.1).

EXTENT_ML. The likelihood of a source being extended is included for extended and newly also for point-like sources in all 4XMM catalogues. It is derived from a fit with a β-profile broadened PSF and calculated with respect to the null hypothesis that the source is point-like as the log-likelihood difference between the extended and the point-like fits using Cash statistics. The minimum allowed core radius of extended sources is 6″. For negative values, thus, the source is detected with higher significance in the fit with a point-like PSF.

PN_PILEUP, M1_PILEUP, M2_PILEUP indicate whether the source might be affected by pile-up in the pn, MOS1, MOS2 exposure. The method to derive the values is described in Webb et al. (2020).

N_BLEND gives the number of simultaneously fitted sources in emldetect. In the catalogue pipeline, it is limited to two.

DIST_REF is the distance of a source to the reference coordinates in the centre of a stack. It is given in arcminutes.

The flag OVERLAP indicates whether the centre of the source is located in the overlap area of two or more observations. It corresponds to N_CONTRIB > 1 in the summary row and is set in all rows of a source. In rare cases, for instance on chip gaps, the centre of the source may be covered by one observation only, triggering the flag to be false, while the outer regions are covered by several observations.

EP_FLAG. The first nine characters of the all-EPIC quality flag include the automatically set flags as in the previous catalogue. A tenth character adds the manual flag from visual screening (Sect. 4.2), being “T” if the source has been regarded obviously spurious and “F” otherwise.

STACK_FLAG. The integer summary of the quality assessment takes the values 0 to 3 from the automatically set flags as introduced by Traulsen et al. (2019) and summarised in Sect. 4.2. If a manual flag has been set, STACK_FLAG is increased by ten.

ASTCORR. This flag indicates whether an exposure was astrometrically corrected before performing stacked source detection. It is thus set in the observation-level rows and undefined in the summary rows. If it is true, the following astrometry-related columns are set, whose prefix CC_ stands for the task catcorr:

CC_RAOFFSET, CC_DEOFFSET give the field shift in arcseconds as derived by the SAS task catcorr for 4XMM-DR9 (Webb et al. 2020 and Sect. 2.3). CC_ROT_CORR gives the catcorr field rotation in degrees. CC_POFFSET is the absolute offset between the original and the corrected source position in arcseconds. CC_RAOFFERR, CC_DECOFFERR, CC_ROT_ERR, and CC_POFFERR are the corresponding 1σ errors.

CC_REFCAT is the name of the reference catalogue used by catcorr. CC_NMATCHES gives the number of matches between 4XMM-DR9 detections in the field and sources in the reference catalogue which were used by catcorr to determine the field rotation and shift.

The following six columns inform on the background level of an observation (Sect. 3.1). They are thus set in the observation-level rows and undefined in the summary rows:

PN_BKG_CRAREA, M1_BKG_CRAREA, M2_BKG_ CRAREA give the instrument-specific background rate per square arcsecond in the source-free regions. They are derived per exposure and thus identical for all sources in a given exposure.

PN_BKG_CPROB, M1_BKG_CPROB, M2_BKG_CPROB are the respective Cauchy probabilities. They are derived from the rate distributions per observing mode and filter (Sect. 3.1).

*_4XMMDR9. Columns copied from 4XMM-DR9 have the suffix _4XMMDR9 and are set if a 4XMM-DR9 source has been associated to a DR9s source (Sect. 4.3). They correspond to the 3XMM-DR7 columns described by Traulsen et al. (2019). A new column SUM_FLAG_MIN_4XMMDR9 gives the lowest – best – summary flag of all individual detections which are merged into the 4XMM-DR9 source.

The integer OBS_CLASS indicates the usable area of an observation in 4XMM-DR9. It is thus set in the observation-level rows and undefined in the summary rows. In 3XMM-DR7s, it was included in the list of observations only. The definition of the OBS_CLASSes is detailed by Webb et al. (2020).

4.5. Auxiliary products and catalogue access

The catalogue is published in the Flexible Image Transport System (FITS) format at the web pages of the XMM-Newton SSC at IRAP, Toulouse¹¹, at AIP, Potsdam¹², and at the VizieR ftp archive¹³ hosted by the Centre de Données astronomiques de Strasbourg (CDS). Searchable interfaces are provided by VizieR¹⁴, the XMM-Newton Science Archive (XSA)¹⁵, and the HEASARC service¹⁶. The online documentation¹⁷ and the list of observations in FITS format and as an HTML table are available from the IRAP and AIP web pages.

Several auxiliary products are produced alongside the catalogue and published online. For each stack, all-EPIC mosaic images in the five XMM-Newton standard energy bands and in the full 0.2−12.0 keV energy range, two full-band exposure maps, and three types of coverage maps are published at the AIP web page¹⁸. For each catalogue source, three auxiliary images are provided: An 0.2−12.0 keV full-band X-ray image covering a 10′ × 10′ region centred at the source position, a false-colour X-ray image of the same region, and an optical finding chart covering a 2′ × 2′ region. For each source with at least two observations, an X-ray light curve is provided. All auxiliary images and light-curve plots are available from the XSA interface. Information on their production, the changes for 4XMM-DR9s with respect to the previous versions, and example images are included in Appendices A.1 and A.2.

Upper flux limits for various high energy missions are provided by the XMM-Newton SOC in an upper-limit service on the XMM-Newton web pages¹⁹. The upper limit to the source flux at a given position is calculated from pipeline processed images, exposure maps, and background maps (König et al., in prep.). Stacked data for 4XMM-DR9s have been made available to include upper limit fluxes in overlapping images.

5. Catalogue characterisation and long-term variability of sources

5.1. Astrometric accuracy

The median statistical position error $\mathrm{RADEC}\_\mathrm{ERR}/\sqrt{2}$ of 4XMM-DR9s is 0.8″ for point-like sources, 1.6″ for extended sources in overlap areas, and 2.3″ for extended sources in uniquely observed areas. To validate their positional accuracy, we compare them to the better constrained positions of SDSS-DR12 quasars (Alam et al. 2015, for the method cf. Traulsen et al. 2019). From the best matches within a radius of 15″ between all sources in both catalogues, 6118 spectroscopically confirmed quasars are selected. Their offsets in units of the 1σ-error are shown in Fig. 14. Ideal positions and position errors are expected to be Rayleigh distributed. Deviations from a Rayleigh distribution indicate that the position errors are not purely statistical, but include systematics for example from uncertainties in the boresight calibration. An additional error component is thus added quadratically to the statistical position errors as ${\sigma }_{\text{combined}}={({\sigma }_{\text{stat}}^2+{\sigma }_{\text{add}}^2)}^{0.5}$ and varied to find the best agreement with a Rayleigh distribution. The best fit is achieved with σ_add = 0.227″. To compare with 3XMM-DR7s, we also determine a linearly added error component σ_stat + σ_lin. In a slightly poorer fit than the quadratic term, it evaluates to 0.064″. The results are essentially the same when reducing the matching radius to 5″. The error-normalised offsets peak neatly at one, but show similar deviations from an ideal Rayleigh distribution as the other serendipitous catalogues (e.g. Rosen et al. 2016; Webb et al. 2020): a lower maximum and a broader tail, suggesting that part of the EPIC error might be underestimated.

Fig. 14. Error-normalised position offsets between 4XMM-DR9s and 6118 SDSS-selected quasars compared to an ideal Rayleigh distribution (red). The histograms show the distributions for the pure statistical position error σstat (blue) and for σcombined (black) which includes the additional error component σadd = 0.227″.

When working with large images in tangential projections, potential distortions have to be taken into account. The catalogue provides the distance of a source to the reference coordinates of the stack in the column DIST_REF. The largest image in the catalogue is almost 10° wide, and the largest offset of a source from the stack centre is 4.8°. Since the PSFs are chosen for azimuth and off-axis angle with respect to the aim point of the individual observation and fitted in sub-images, projection effects are negligible anyway. Source positions in 4XMM-DR9 and DR9s show no systematic deviation from each other. In particular there is no offset as a function of DIST_REF.

5.2. Comparison to 4XMM-DR9

In the observations that were used to compile 4XMM-DR9s, 4XMM-DR9 records 282 804 unique sources²⁰. Almost 85% of the DR9 sources have at least one contributing detection with a good quality flag 0 or 1. Applying the matching strategy of Sect. 4.3, we find DR9 counterparts for 214 170 sources in 4XMM-DR9s. For 193 DR9s sources, more than one DR9 source was located within the matching radius and the closest match chosen. A total of 74 021 4XMM-DR9s sources have no association. Part of this increase is regarded a benefit of the depth of the repeated observations. Another part, typically close to the detection limit, appears as a consequence of the astrometric correction applied prior to stacked source detection and the different image binning. We address these aspects by firstly analysing the source content of the two catalogues in non-overlap areas to then estimate the increase of sources in the overlapping areas thanks to stacking them.

In regions observed only once, DR9s and DR9 represent two source detection runs on the shifted and un-shifted event lists. Out of the 65 307 un-flagged DR9s sources with just one contributing observation, 9018 were not associated to a 4XMM-DR9 source. 7620 of them are point-like and 1758 are extended sources with a larger positional uncertainty than point-like sources. The detection likelihoods and counts of the point-like un-flagged sources are shown in Fig. 15. Most of the non-matches are close to the detection limit with low detection likelihood and few counts: 66% have detection likelihoods below 10 and 85% below 20. Close to the detection limit, differences between DR9s and DR9 source lists mainly arise from the astrometric correction, which is applied to the observations before running stacked source detection for DR9s and only after source detection on the individual observations for DR9. The different reference coordinates for the image binning, which are common coordinates for all observations in a stack for DR9s versus the individual aim point of each observation for DR9, may also slightly influence the fit results in particular near the detection limit. Both astrometric correction and reference coordinates alter the centring of the image bins and cause a slightly different distribution of the photons over the pixels. This affects the PSF fit and thus the detection likelihood. Changes in the likelihood can result in gain or loss of sources close to the detection limit.

Fig. 15. Parameters of 4XMM-DR9s-only sources with one contributing observation. Upper panels: detection likelihoods (panel a) and source counts (panel b) of N_CONTRIB = 1-sources without (blue) and with (red) a DR9 association. Lower panels: panel c: detection likelihoods of DR9s-only sources with N_CONTRIB = 1 in good (blue) and complex (red) sky areas according to the DR9 quality index OBS_CLASS. Panel d: detection likelihoods of DR9s-only sources with N_CONTRIB = 1 (blue) compared to DR9s-only sources in overlap areas (red). All distributions are normalised to the sample size.

A further reason for non-matches between DR9 and DR9s lies in the pre-selection of sources before matching them. DR9 sources with a poor quality flag SUM_FLAG ≥ 2 were not included in the matching exercise described in Sect. 4.3 and thus not listed in 4XMM-DR9s. We investigate the effect of this restriction in a second match between all un-matched DR9s and DR9 sources, now using all DR9 sources irrespective of their quality flag. The distance between matches is now limited to a maximum of 1′ to reduce false associations of sources with large positional errors. More than 80% of the DR9s-only sources with STACK_FLAG ≤ 1 are close to a DR9 source with SUM_FLAG ≥ 2.

In addition, a significantly higher percentage of un-matched than of matched DR9s sources is subject to source confusion, and many un-matched sources are located in problematic observations for example with high X-ray background, bright single-reflection patterns, or very complex extended structures, where the source position can be determined less precisely. A non-quantified fraction of sources will thus be lost through the matching radius. Figure 15c shows the likelihood distributions for different observation qualities. At detection likelihoods above 20, about 40% of the un-matched sources are in OBS_CLASS ≤ 3 observations compared to 9% of the matched sources. Visual inspection confirms that bright sources without a DR9 association, which can be seen in Fig. 15b, are typically located in such problematic regions, or their potential counterpart in DR9 was flagged with SUM_FLAG ≥ 2 and therefore not used in the match. Thus, sources in non-overlapping areas come without a DR9 match through various reasons and partly combinations of them: different image binning and the corresponding fitting effects, poor observing conditions, complex X-ray structures, source confusion, asymmetric use of flagged sources in the match not to contaminate DR9s with questionable DR9 associations.

Finally, we investigate the DR9s-only sources in overlap regions (N_CONTRIB > 1), hence in genuine stacking areas. Compared to non-overlapping areas, the source density is increased thanks to the higher cumulated exposure time (cf. Fig. 10). Through stacked source detection, the source positions are better constrained, which gives more reliable matching results. In those sky areas, about 20% of the DR9s sources with the best quality flag STACK_FLAG = 0 have no counterpart in DR9. The fraction of DR9s-only sources is thus about twice as high in overlapping areas than in non-overlap areas. The un-flagged DR9s-only sources tend to have higher detection likelihoods than in non-overlap areas (Fig. 15d), a larger fraction is found in regions without source confusion, and the fraction of DR9s-only sources in observations with poor OBS_CLASS ≥ 3 is lower than in non-overlap areas. Eighty-two percent of the DR9s-only sources involve at least one good observation with OBS_CLASS ≤ 2, compared to 94% of the sources with a DR9 association. If including 4XMM-DR9 sources in the match which only have detections with problematic quality flags SUM_FLAG ≥2 as described in the previous paragraph, another 3426 potential DR9 counterparts to DR9s sources with STACK_FLAG ≤1 are found, still leaving 18% of them un-matched (22% of all DR9s sources and quality flags). The high fraction of DR9s-only sources in overlap areas is thus regarded being mostly an effect of the higher sensitivity through stacking.

5.3. Long-term variability

Variability studies directly from stacked source detection have major advantages over individual detections: Fluxes are determined at the same position in all exposures. The total flux is derived in the simultaneous fit without need to match single detections, which might involve false associations. And lastly, the fluxes are determined for any detection likelihood in the individual observation. About 12% of the observation-level fluxes in 4XMM-DR9s are derived for a source likelihood below the detection limit, thus for a non-detection in the respective individual observation. Fake variability however can still rarely arise, for example from hot pixels during an observation or spurious detections close to a bright source. Quality filtering based on the flags provided in the catalogue is thus essential and visual inspection of the auxiliary source images delivered with the catalogue is recommended in searches for so far unknown long-term variability between observations.

For each catalogue source with at least two valid measurements, the variability parameters defined by Traulsen et al. (2019) are provided. They are calculated from the stacked and observation-level fluxes in each energy band F_k, where k runs from 1 to n observations (columns EP_e_FLUX in the catalogue, e = 1...5), from the total flux F_EPIC (column EP_FLUX in the catalogue), and from their respective 1σ errors: the reduced χ² of long-term flux changes

$$\begin{array}{c}\hfill \mathrm{VAR}\_\mathrm{CHI}2=\frac{1}{n-1}{\displaystyle \sum _{k=1}^n{\left(\frac{F_k-F_{\mathrm{EPIC}}}{{\sigma }_k}\right)}^2,}\end{array}$$(1)

the associated cumulative chi-square probability of the flux measurements being consistent with constant flux

$$\begin{array}{c}\hfill \mathrm{VAR}\_\mathrm{PROB}={\displaystyle {\int }_{{\chi }^2}^{\infty }\frac{x^{\nu /2-1}{e}^{-x/2}}{2^{\nu /2}\mathrm{\Gamma }(\nu /2)}\mathrm{d}x,}\end{array}$$(2)

where smaller values indicate a higher chance that the source is variable and Γ denotes the gamma function, the ratio between the highest and lowest observation-level flux

$$\begin{array}{c}\hfill \mathrm{FRATIO}=F_{\max }/F_{\min },\end{array}$$(3)

the associated 1σ error

$$\begin{array}{c}\hfill \mathrm{FRATIO}\_\mathrm{ERR}={(\frac{{\sigma }_{F\min }^2}{F_{\min }^2}+\frac{{\sigma }_{F\max }^2}{F_{\max }^2})}^{0.5}\frac{F_{\max }}{F_{\min }},\end{array}$$(4)

and the largest flux difference between any combination of the observation-level fluxes in terms of σ

$$\begin{array}{c}\hfill \mathrm{FLUXVAR}=\underset{k,l\in [1,n]}{max}\frac{|F_k-F_l|}{\sqrt{{\sigma }_k^2+{\sigma }_l^2}}\end{array}$$(5)

where both k and l cover the indices of the n observations of the source.

For a thorough variability analysis, several variability parameters should be studied jointly. Here, we concentrate on the all-EPIC parameter VAR_PROB only, which is the probability that the mean fluxes per observation are consistent with constant long-term behaviour. 7 182 un-flagged sources are likely long-term variable with VAR_PROB ≤ 10⁻⁵ (3.75% of all multiply observed un-flagged sources), and 11 327 (5.91%) have VAR_PROB ≤ 10⁻³. Almost 90% of the variability candidates in 4XMM-DR9s have an association in 4XMM-DR9, but about 20% of them with only one valid DR9 detection although covered by several observations. For the latter ones, stacked source detection is the only way to investigate the long-term evolution of their fluxes, which are derived for all stacked observations and do not depend on the detectability of a source in the individual observation as in DR9. In DR9, short-term intra-observation variability is determined by means of a χ² variability test on the light curves of a detection in each instrument against the null-hypothesis that the brightness fluctuations are consistent with constant source flux (Rosen et al. 2016; Webb et al. 2020). A source is considered short-term variable if the probability derived from the χ² test is 10⁻⁵ or lower in at least one instrument in at least one observation. The light curves of 26.5% of the unique 4XMM-DR9 sources have enough good time bins to perform the χ² test. Among the DR9 associations of long-term variable 4XMM-DR9s sources with VAR_PROB ≤ 10⁻⁵, 14% are marked as short-term variable as well, while 54% show no sign of short-term variability. For the rest, the short-term probability could not be determined.

The long-term light curves of the most variable 4XMM-DR9s objects show various kinds of variability: for example flaring or on-off behaviour, flux changes of the order of days, continuous brightening or darkening of sources. Figure 16 includes example light curves from 4XMM-DR9s. In the following subsections, we illustrate the potential of DR9s variability in case studies of object classes and X-ray surveys.

Fig. 16. Example long-term light curves of variable sources in 4XMM-DR9s. Red: mean all-EPIC flux and flux error. Blue: observation-level fluxes. The plot symbols code the variability. Information on short-term variability within an observation is inherited from 4XMM-DR9. Filled circle: variability. Open circle: no variability. Box: short-term variability unknown. Dot: non-detection in 4XMM-DR9. From top to bottom: flaring M dwarf, star of unknown optical variability, gamma-ray burst afterglow, which became undetectable in single XMM-Newton observations after about two years, and a quasar.

5.3.1. Variability of SIMBAD- and SDSS-classified sources

More than 80 000 DR9s un-flagged point sources in 4XMM-DR9s have a tentative counterpart in SIMBAD (Wenger et al. 2000) or SDSS-DR12 (Alam et al. 2015) within 3″. Among them, 3779 have VAR_PROB ≤ 10⁻⁵ in 4XMM-DR9s, compared to a total of 7182 un-flagged variable DR9s sources (Table 1). The pure position match results in about 5% false associations among the variable sources (for the method cf. Traulsen et al. 2019). We use the matched sources to investigate the variable content of different source classes. For ten source classifications in SIMBAD and – if available – in SDSS, the number of variables and their share among all matched class members are shown in Fig. 17: stars without the following sub-categories, high-proper motion stars, young stellar objects and T Tauri stars, non-interacting binaries, interacting binaries, galaxies without AGN and QSOs, AGN without QSOs, QSOs, and un-classified X-ray and γ-ray sources. In few cases, high-proper motion stars can mimic long-term variability in 4XMM-DR9s, if they are fitted as several detections at different positions. In total, 652 4XMM-DR9s sources are matched with a high-proper motion star in SIMBAD, and 41 have VAR_PROB ≤ 10⁻⁵, which is the same proportion as for all stars.

Fig. 17. SIMBAD and SDSS categories of 3 343 tentative counterparts to 4XMM-DR9s variables with VAR_PROB ≤ 10−5. Upper panel: total number of matches. Lower panel: percentage of variables among all matched variable and non-variable class members.

From the matching sample, we select sources with information on long-term and on short-term variability. They have at least three 4XMM-DR9s observations that cover a minimum time span of 30 days and a 4XMM-DR9 association with a sufficient number of counts to determine intra-observation variability. Analogously to 4XMM-DR9, sources are considered short-term variable for CHI2PROB_4XMMDR9 ≤ 10⁻⁵ and long-term variable for VAR_PROB ≤ 10⁻⁵. Figure 18 gives the share of long- and short-term variables among their classes in the subsample. Three source classes are included, which are expected to have different fractions of long- and short-term variable members: stars, which can be variable on all time scales; interacting binaries, for part of which orbital modulations can be recovered within an observation and for part of which long-term flux changes for example due to accretion-rate changes or nova eruptions are detected; and quasars, which are predominantly variable on longer time scales, if at all.

Fig. 18. Long-term inter-observation and short-term intra-observation variability for SIMBAD- and SDSS- selected source classes: from left to right all classified sources (black), all variable stars without binaries (blue), interacting binaries (red), and quasars (green).

5.3.2. Variable 4XMM-DR9s sources in the XMM-Newton slew and eROSITA surveys

The XMM-Newton slew survey (Freyberg et al. 2006), performed while slewing the telescope between targets, has reached a sky coverage of about 84% as of March 2017 and an EPIC pn sensitivity of about 6 × 10⁻¹³ erg cm⁻² s⁻¹ in the soft energy band²¹, 3 × 10⁻¹² erg cm⁻² s⁻¹ in the hard energy band, and 1 × 10⁻¹² erg cm⁻² s⁻¹ in total (Saxton et al. 2008; Warwick et al. 2012). To investigate its potential for combined variability studies, we derive the fluxes of the 4XMM-DR9s sources in the slew energy bands and select the bright objects whose soft, hard, or total all-EPIC or instrument-level flux is above the slew sensitivity. For them, slew observations can contribute to the long-term variability information. Among 1596 bright enough point-like and un-flagged sources, more than 700 are in the footprint of the slew survey and covered up to 13 times by slew exposures. 497 have a tentative counterpart within a 15″ matching radius in the full Slew Survey Source Catalogue XMMSL2 and 444 in its so-called clean edition, from which detections with low detection likelihood and detections with poor quality flags were removed. 358 matches with the full XMMSL2 are located in overlap regions of DR9s and 139 in regions observed once, increasing the pool of sources that can be used in XMM-Newton variability studies. From all DR9s and XMMSL2 fluxes of a source, we derive a combined VAR_PROB_comb that all measurements are consistent with constant flux. Additional 28 sources show signs of long-term variability according to VAR_PROB_comb ≤ 10⁻⁵, which are not detected as long-term variable in DR9s alone: 22 with only one DR9s observation and 6 with two DR9s observations, which were taken between one day and 12 years apart from each other.

The Russian-German Spectrum Roentgen Gamma mission (SRG) was launched in July 2019 and surveys the X-ray sky with its two telescopes eROSITA and ART-XC for four years. eROSITA (Merloni et al. 2012) covers a similar energy range as XMM-Newton/EPIC and commenced its survey end of 2019. Synergies with XMM-Newton include in particular analyses of the long-term behaviour of X-ray sources targeted by both missions. To estimate the common source content of 4XMM-DR9s and future eROSITA catalogues, we derive an all-sky sensitivity map from the exposure forecast for eROSITA’s first six-months survey (eRASS1) and for the total four-year survey (eRASS8), which are available to the eROSITA consortium. We then compare the EPIC fluxes of each 4XMM-DR9s source to the map values at the source position. Thirty-four percent of the sources have a flux above the eRASS1 limiting sensitivity in total or at least in one contributing observation. The fraction increases to 75% in eRASS8 after completing eROSITA’s four-year survey, forming a valuable resource for cross-mission studies.

6. Summary and conclusions

4XMM-DR9s is the second serendipitous source catalogue from overlapping observations, based on the so far largest sample of exposures. They were selected if they overlap by at least 1′ in radius and have a background level below a 92% Cauchy probability as derived in Sect. 3.1. Thanks to an event-based astrometric correction, which was applied at the beginning of the processing, the positional accuracy of the sources has been clearly improved compared to source detection in the uncorrected observations.

In addition to the automated source flagging by the task dpssflag, the catalogue source lists were screened visually and obviously spurious detections marked manually. Both processes cannot be complete, and an un-flagged catalogue sample cannot be expected to be free from bad detections. But source selection based on the quality flags reduces the spurious content of the catalogue significantly.

From 6604 observations in 1329 stacks, 288 191 unique sources were extracted in total, 218 283 of them multiply observed. Additional sources were detected compared to source detection on individual observations, and the source parameters can be better constrained in overlap areas. Long-term inter-observation variability is investigated directly based on the source-detection fit without need to match detections from different observations. Thanks to the simultaneous fit to all observations in a stack, 320 590 new flux determinations and flux errors are available for 106 127 sources with a 4XMM-DR9 association, which could not be detected in part of the individual DR9 observations. 9912 4XMM-DR9s sources still come without a measurable flux in one or more contributing observations because of zero counts in the fit region. Upper flux limits can be retrieved from the upper-limit server of the SOC.

The flux determinations from the simultaneous source detection fits let us directly derive variability parameters for all catalogue sources with at least two valid measurements. About 6% of them show signs of at least moderate (VAR_PROB ≤ 10⁻³) and 4% of high (VAR_PROB ≤ 10⁻⁵) long-term variability. Only part of them are known to be short-term variable as well, and about a third have no detection or only one detection in 4XMM-DR9. Their fluxes and long-term variability were measured for the very first time in 4XMM-DR9s. The catalogue from overlapping observations thus serves as a large data base for cross-matching and for long-term studies of X-ray emitting objects, also in the context of new and future missions like eROSITA and Athena, ESA’s proposed future X-ray observatory carrying a high-resolution spectrograph and a wide-field imager, which was selected by ESA within its Cosmic Vision programme (Nandra et al. 2013).

The next catalogue releases in the 4XMM series will concentrate on new public observations. While the standard 4XMM catalogue of detections can be incremented by adding the new detections, a refined strategy is needed for the stacked catalogue. The additional observations may form new stacks which can be included directly, they may become part of existing stacks, or may even combine previously independent stacks. Therefore, source detection will be (re-)run on all old and new observations contributing to the modified sky area of the newly designed stacks. Sources from those sky areas contained in the current catalogue will be replaced with those from the new run. It is foreseen to publish the catalogue updates on an approximately yearly basis.

Acknowledgments

We thank our anonymous referee for the useful comments. The support of SSC work at AIP by Deutsches Zentrum für Luft- und Raumfahrt (DLR) through grants 50 OX 1701 and 50 OX 1901 is gratefully acknowledged. We particularly appreciate the close collaboration with the colleagues at ESA’s XMM-Newton Science Operations Centre (SOC) and the support by the CDS team. The French teams are grateful to Centre National d’Études Spatiales (CNES) for their outstanding support for the SSC activities. FJC acknowledges financial support through grant AYA2015-64346-C2-1P (MINECO/FEDER). FJC and MTC acknowledges financial support from the Spanish Ministry MCIU under project RTI2018-096686-B-C21 (MCIU/AEI/FEDER/UE), cofunded by FEDER funds and from the Agencia Estatal de Investigación, Unidad de Excelencia María de Maeztu, ref. MDM-2017-0765. This project has made use of CDS services and the SIMBAD database, operated at CDS, Strasbourg, France, of FTOOLS by NASA’s HEASARC (Blackburn 1995), of TOPCAT/STILTS (Taylor 2005), and of SDSS-III. Funding for SDSS-III has been provided by the Alfred P. Sloan Foundation, the Participating Institutions, the National Science Foundation, and the US Department of Energy Office of Science. The SDSS web site is www.sdss.org. SDSS is managed by the Astrophysical Research Consortium for the Participating Institutions of the SDSS Collaboration including the Brazilian Participation Group, the Carnegie Institution for Science, Carnegie Mellon University, the Chilean Participation Group, the French Participation Group, Harvard-Smithsonian Center for Astrophysics, Instituto de Astrofísica de Canarias, The Johns Hopkins University, Kavli Institute for the Physics and Mathematics of the Universe (IPMU)/University of Tokyo, the Korean Participation Group, Lawrence Berkeley National Laboratory, Leibniz Institut für Astrophysik Potsdam (AIP), Max-Planck-Institut für Astronomie (MPIA Heidelberg), Max-Planck-Institut für Astrophysik (MPA Garching), Max-Planck-Institut für Extraterrestrische Physik (MPE), National Astronomical Observatories of China, New Mexico State University, New York University, University of Notre Dame, Observatário Nacional/MCTI, The Ohio State University, Pennsylvania State University, Shanghai Astronomical Observatory, United Kingdom Participation Group, Universidad Nacional Autónoma de México, University of Arizona, University of Colorado Boulder, University of Oxford, University of Portsmouth, University of Utah, University of Virginia, University of Washington, University of Wisconsin, Vanderbilt University, and Yale University.

References

Appendix A: Auxiliary information on the catalogue

A.1. Auxiliary images and light curves

Auxiliary images in PNG format are published for each 4XMM-DR9s source: X-ray images from the 4XMM-DR9s pipeline and an optical finding chart chosen from ESO Online Digitized Sky Survey, Pan-STARRS G, and skyMapper G. An X-ray light-curve plot is published for each multiply observed source. The production of these data is detailed in Traulsen et al. (2019). The following updates were applied for 4XMM-DR9s: The full-band X-ray image now shows all detected sources in the plotted region with dotted white marks in addition to the central source which is marked in blue. As before, diamonds with a fixed size are used for point-like sources and circles with the same radius as the core of the extent model for extended sources. In the light curves, an additional plot symbol has been introduced besides the open circle for sources with a VAR_PROB above 0.01 and a filled circle for VAR_PROB ≤ 0.01: If a source was fitted with zero counts for one out of two observations, VAR_PROB remains undefined. The single valid flux value is marked by an x in these cases. If more than two observations are involved, the variability parameters are derived from those with defined fluxes. The flux axis is adjusted to minimum and maximum flux of the source including error bars, covers at least one magnitude, and has an absolute minimum of 10⁻¹⁷ erg cm⁻² s⁻¹. Figures A.1 and A.2 include an example set of the light curves and images that are produced for each 4XMM-DR9s catalogue source and available from the XSA interface (Sect. 4.5).

Fig. A.1. Example of the auxiliary light curves produced for 4XMM-DR9s sources and published in the XSA.

Fig. A.2. Example of the auxiliary images produced for 4XMM-DR9s sources for the object of Fig. A.1: full-band X-ray image (upper panel), false-colour X-ray image (middle panel), and optical finding chart (lower panel).

A.2. Auxiliary data

Our catalogue pipeline generates auxiliary data in FITS format for each 4XMM-DR9s stack, including composite images, exposure maps, and coverage maps of the stack. They are published via xmmssc.aip.de. All-EPIC mosaic images are generated for the five XMM-Newton standard energy bands and for the full 0.2−12.0 keV band. They are the sum of all observation- and instrument-level images.

Two exposure maps include the total exposure at a given pixel, which is summed over all contributing observations. For each observation, the maximum exposure time of the three EPIC instruments per pixel is chosen. One exposure map gives the total un-vignetted exposure. The other map is created from exposure maps that are multiplied with the respective vignetting factor, which depends on the instrument, the detector coordinates, and the energy band.

Three coverage maps are created from the detection masks of the individual exposures. The maps named “cov” show the valid pixels in a stack, “1” indicating the exposed pixels and “0” the un-exposed pixels. The maps “nobs” give the number of overlapping observations and “nexp” the number of overlapping exposures in each pixel, respectively. Figure A.3 shows three example sets of coverage masks for different patterns of overlaps.

Fig. A.3. Examples of the coverage maps of 4XMM-DR9s stacks: exposed and un-exposed pixels (left panels), number of overlapping observations (middle panels), and number of overlapping exposures (right panels).

A.3. Catalogue layout

Several sets of source parameters are derived from the simultaneous source-detection fits (Sect. 2). They are listed in the catalogue in 310 parameter columns and in several rows for each detected source. The first row includes the stack summary, i.e. the parameters derived from all images and all observations involved. In the following rows, source parameters are given for each contributing observation that was considered in the fit. They are derived from the subset of images taken during this specific observation. Since some parameter columns are only defined in the stack-summary rows and some only in the observation-level rows, these columns can be used to identify them. Stack-summary rows can be selected for example through their valid N_OBS and N_CONTRIB values. Observation-level rows are identified for example through their OBS_ID and REVOLUT entries. Sources with a match in 4XMM-DR9 are characterised by a valid SRCID_4XMMDR9 or their URL_4XMMDR9. At observation level, they have a valid DETID_4XMMDR9. Figures A.4 and A.5 show usage examples of the FITS table in a FITS viewer (here: TOPCAT) and of the information in the online interface XSA.

Fig. A.4. Screen shot of the 4XMM-DR9s FITS table as displayed by TOPCAT. Stack-summary rows have been selected using the expression “N_CONTRIB > 0” and are marked in blue. Observation-level rows can be identified through their OBS_ID for example.

Fig. A.5. Screen shot of a 4XMM-DR9s search result in the XSA. The sources are listed with their 4XMMs IAU name (“Stacked Cat. Name”) and, if available, with the 4XMM IAU name of their counterpart in 4XMM-DR9. A mouse click on the plus sign opens the list of contributing observations with an option to download their XSA data, as shown for the first and seventh source in this example.

Table A.1 summarises the 310 catalogue columns. Longer descriptions of the columns introduced with 3XMM-DR7s can be found in Traulsen et al. (2019) and of the new and updated 4XMM-DR9s columns in Sect. 4.4 of this paper.

All Tables

All Figures

Fig. 1. Illustration of the PSF shapes in the three EPIC instruments at different off-axis and azimuth angles in overlapping observations and of the parameter determination in the simultaneous maximum-likelihood PSF fit. The normalised PSFs – shown here for 1 keV in a linear intensity scale – are scaled to the count rate in the respective image within the detection fit. Position and extent of a source are the same in all observations, while the count rates are fitted in each image. Observation-level and stacked parameters are derived from the parameters of the contributing images (Traulsen et al. 2019).

In the text

	Fig. 2. catcorr results for the input fields to 4XMM-DR9s which could be astrometrically corrected: histograms of the applied shifts along right ascension (left, red crosses, ΔRA × cos(Dec)) and declination (left, blue pluses, ΔDec), the additional field rotation (right), and the resulting absolute field shift (left, black).
In the text

	Fig. 3. Distribution of the angular distances, given in units of the statistical error, between X-ray and optical SDSS positions before (blue) and after (red) event-based field rectification in a test sample of 25 stacks. The grey curve shows an ideal Rayleigh distribution.
In the text

	Fig. 4. MOS2 background levels between 0.2 and 12.0 keV for the observing modes full frame, large window, small window, and timing in thin filter (upper panel) and for the thin, medium, and thick filter in full-frame mode (lower panel). The dashed and dotted lines show the upper part of the corresponding Cauchy probability values for the different filters.
In the text

	Fig. 5. Sky map of the 4XMM-DR9s input observations in a Hammer-Aitoff projection in Galactic coordinates. The radius of the plot symbols corresponds to 2°, thus to four times the actual field-of-view of XMM-Newton. Colour codes the total exposure time of the overlapping observations in each plotted bin.
In the text

	Fig. 6. Distribution of observation modes, filters, and usable chip area OBS_CLASS in 4XMM-DR9s. Possible modes are extended full frame (EF), full frame (FF), large window (LW), small window (SW), timing (TI). Filters include thin (TN), medium (ME), and thick (TK). Ten observations have the poorest OBS_CLASS 5.
In the text

	Fig. 7. Number of observations in the 4XMM-DR9s stacks.
In the text

	Fig. 8. Cumulative sky area of multiply observed patches over exposure time in the input sample. “All-EPIC” shows the total exposure, for which the maximum exposure among the active EPIC instruments is determined in each observation and sky pixel and then summed over all observations.
In the text

Fig. 9. Example of a stack with six observations, targeting the star cluster NGC 2264. Upper panel: false-colour image in three energy bands 0.2 − 1.0, 1.0 − 2.0, and 2.0 − 12.0 keV to illustrate the different flux maxima of the detected objects. Lower panel: exposure map, giving the total exposure in ks, taking vignetting into account, with 4XMM-DR9s sources as red circles and 4XMM-DR9 sources as blue diamonds. The symbol size scales with the core radius of the extent model. Several low-quality (flagged) DR9 detections were not recovered in DR9s.

In the text

	Fig. 10. Source density in 4XMM-DR9s over number of contributing observations (left panel) and total exposure time in all contributing observations, taking vignetting into account (right panel). Only small sky areas are covered by more than 30 observations or for more than 0.5 Ms (cf. Fig. 8), causing a wider scatter in the distributions.
In the text

	Fig. 11. Flux distribution of multiply observed sources (blue) and of sources covered once (red) in 4XMM-DR9s. Upper panel: histograms normalised to the sample size. Lower panel: cumulative histograms.
In the text

	Fig. 12. Parameters of sources that surpass the minimum detection likelihood only in the stack summary. Their observation-level detection likelihoods in the images of a specific observation, which are derived from the same simultaneous fit, are all below the limit of six.
In the text

Fig. 13. Examples from the manual screening process: bad detector features (panel a), single-reflection patterns (panel b), RGA diffraction pattern (panel c), path of a planet in a mosaic-mode observation (panel d). Yellow circles show all detections, where large radii correspond to the core radius of extended sources. Bad regions or detections are marked manually (green). The detections within them (red) are then flagged in the catalogue.

In the text

	Fig. 14. Error-normalised position offsets between 4XMM-DR9s and 6118 SDSS-selected quasars compared to an ideal Rayleigh distribution (red). The histograms show the distributions for the pure statistical position error σstat (blue) and for σcombined (black) which includes the additional error component σadd = 0.227″.
In the text

Fig. 15. Parameters of 4XMM-DR9s-only sources with one contributing observation. Upper panels: detection likelihoods (panel a) and source counts (panel b) of N_CONTRIB = 1-sources without (blue) and with (red) a DR9 association. Lower panels: panel c: detection likelihoods of DR9s-only sources with N_CONTRIB = 1 in good (blue) and complex (red) sky areas according to the DR9 quality index OBS_CLASS. Panel d: detection likelihoods of DR9s-only sources with N_CONTRIB = 1 (blue) compared to DR9s-only sources in overlap areas (red). All distributions are normalised to the sample size.

In the text

Fig. 16. Example long-term light curves of variable sources in 4XMM-DR9s. Red: mean all-EPIC flux and flux error. Blue: observation-level fluxes. The plot symbols code the variability. Information on short-term variability within an observation is inherited from 4XMM-DR9. Filled circle: variability. Open circle: no variability. Box: short-term variability unknown. Dot: non-detection in 4XMM-DR9. From top to bottom: flaring M dwarf, star of unknown optical variability, gamma-ray burst afterglow, which became undetectable in single XMM-Newton observations after about two years, and a quasar.

In the text

	Fig. 17. SIMBAD and SDSS categories of 3 343 tentative counterparts to 4XMM-DR9s variables with VAR_PROB ≤ 10−5. Upper panel: total number of matches. Lower panel: percentage of variables among all matched variable and non-variable class members.
In the text

	Fig. 18. Long-term inter-observation and short-term intra-observation variability for SIMBAD- and SDSS- selected source classes: from left to right all classified sources (black), all variable stars without binaries (blue), interacting binaries (red), and quasars (green).
In the text

	Fig. A.1. Example of the auxiliary light curves produced for 4XMM-DR9s sources and published in the XSA.
In the text

	Fig. A.2. Example of the auxiliary images produced for 4XMM-DR9s sources for the object of Fig. A.1: full-band X-ray image (upper panel), false-colour X-ray image (middle panel), and optical finding chart (lower panel).
In the text

	Fig. A.3. Examples of the coverage maps of 4XMM-DR9s stacks: exposed and un-exposed pixels (left panels), number of overlapping observations (middle panels), and number of overlapping exposures (right panels).
In the text

	Fig. A.4. Screen shot of the 4XMM-DR9s FITS table as displayed by TOPCAT. Stack-summary rows have been selected using the expression “N_CONTRIB > 0” and are marked in blue. Observation-level rows can be identified through their OBS_ID for example.
In the text

	Fig. A.5. Screen shot of a 4XMM-DR9s search result in the XSA. The sources are listed with their 4XMMs IAU name (“Stacked Cat. Name”) and, if available, with the 4XMM IAU name of their counterpart in 4XMM-DR9. A mouse click on the plus sign opens the list of contributing observations with an option to download their XSA data, as shown for the first and seventh source in this example.
In the text