© ESO 2021

1. Introduction

The INTErnational Gamma-Ray Astrophysics Laboratory (INTEGRAL; Winkler et al. 2003) is a european space agency (ESA) space astronomy mission that has collected data in orbit since 2002. It provides observations of astronomical sources in the keV-MeV energy range. The Imager on Board the INTEGRAL Satellite (IBIS; Ubertini et al. 2003) is a coded-aperture instrument that provides fine imaging (12″ full width half maximum (FWHM)) in a nearly squared partially coded field-of-view (FoV) of 30° ×30°, source identification and spectral sensitivity to both continuum and broad lines between 15 keV and 10 MeV. Its focal plane is composed of two detectors that are optimised for two different energy ranges: the INTEGRAL Soft Gamma-Ray Imager (ISGRI) from 15 to ∼1000 keV (Lebrun et al. 2003) and the Pixellated Imaging CaeSium Iodide Telescope (PICsIT) from 400 keV to 10 MeV (Labanti et al. 2003). In the following, we discuss only the ISGRI detector layer and refer to it as the ISGRI instrument. The Joint European X-Ray Monitor (JEM-X; Lund et al. 2003) provides images, timing, and spectral information in a lower energy range (3–25 keV). It has a narrower circular FoV ∼10° diameter. The SPectrometer of INTEGRAL (SPI; Vedrenne et al. 2003) provides high-resolution spectroscopy data with moderate angular resolution. The SPI instrument is surrounded by an Anti-Coincidence Shield (ACS) that reduces the level of background in SPI and simultaneously works as an all-sky gamma-ray burst (GRB) monitor (Savchenko et al. 2012). The wide FoV of IBIS inherently includes data on a large number of astronomical sources, which are not necessarily the targets of specific observational proposals. This means that all source-specific observational campaigns of INTEGRAL possess a high potential for serendipitous science. All INTEGRAL data become publicly available after a one-year proprietary period, so that an analysis of all the sources found in the FoV is possible.

The astronomical sources visible in X-ray and soft γ-ray band are variable on different timescales from milliseconds to years and decades (van der Klis 2006; Beckmann et al. 2007; Mészáros et al. 2019). In this respect, the 17-year-long data set of INTEGRAL is unique as it contains the information on long-term variability patterns of a large number of sources. Some sources spend long periods in quiescent states, produce little emission, and exhibit occasional outbursts or flares on an irregular basis with duty cycles that are spread over years or decades. The INTEGRAL archive contains data taken nearly continuously since mid-November 2002, with a duty cycle on nearly 90%. The instruments are switched off at each perigee passage through the Earth’s radiation belts. Until March 2003, the on-board settings were continuously adjusted to optimise the performance, making the scientific analysis challenging. The general user is usually suggested to start using archive data after this date, finding nearly 50 mega seconds (Ms) of usable data in the direction of the Galactic centre, the region with the highest exposure. The archive data thus contain information on the previous history of the quiescence and activity of sky sources (> 1200 detected by the imager so far), including those that might undergo outbursts in future.

The INTEGRAL data and offline science analysis (OSA) software that includes data analysis pipelines for all instruments, including ISGRI (Goldwurm et al. 2003) and JEM-X (Westergaard et al. 2003), is distributed by the INTEGRAL Science Data Centre (ISDC, Courvoisier et al. 2003). The main components of OSA have been developed in the period preceding the INTEGRAL launch and are maintained by the ISDC. The architecture of OSA was optimised for computing environments that were common more than 20 years ago. INTEGRAL was initially planned to operate in space for five years and generate relatively small data sets. The only solution for the data processing was via local installation of OSA on a user computer. This is not necessarily the case today for data sets spanning 17 years. Their analysis requires a significant number of computing resources. Moreover, the maintenance of legacy software on evolving operating systems poses increasingly more challenges.

The development of high performance computing (HPC) and cloud computing (CC) technologies and their applications to astronomical data management (Banek et al. 2020; Smith et al. 2019) over the past decades opens up a new possibility of the deployment of OSA-based data analyses without the need for a local installation of the software. It also provides access to a large pool of computing resources, thus significantly reducing the complexity of the analysis of INTEGRAL data. Such an open data analysis (ODA) system for the ISGRI instrument has recently been developed at the department of astronomy of the University of Geneva¹ in synergy with the INTEGRAL Science Data Centre (ISDC)², and it is maintained jointly with the François Arago Centre (FACe) of Astroparticle and Cosmology laboratory in Paris³.

This system entirely relies on the official OSA package, provided and validate by the instrument teams, integrated and distributed by by ISDC. All of the results are not only equivalent, they are identical. This design in principle allows leveraging the unrestrained power of the OSA software and preserving and maintaining the complete potential of the INTEGRAL data for future explorers, without making assumptions about which products are likely to be useful. While this approach may require larger real-time computing resources, the platform exploits a dynamic innovative provenance-based database of pre-computed products to eliminate analysis duplication and flexibly respond to the user requests, as detailed in Sects. 2.2 and 5. This means in principle that the ODA validation is in part redundant because it is equivalent to an OSA validation. On the other hand, this paper reveals how a selection of particular OSA cookbook threads, adopted in ODA, allows us to reproduce the published results.

In the following, we describe the system and characterise its performance through a comparison of science results obtained using the system with previously published benchmark INTEGRAL results on a variety of Galactic and extra-galactic sources. We also demonstrate the usefulness of the system in the context of multi-wavelength studies and discuss advantages and limitations of this online data analysis implementation.

Throughout the paper, we provide direct links to the code we used to produce the figures in the text. We show the potential of this approach for the open access, reusability, and reproducibility of science results.

2. Online data analysis interface

The general scheme of the ODA is shown in Fig. 1. The user has several possibilities to submit requests for analysis of INTEGRAL data. First, through a web browser by accessing the ODA website⁴ on their local computer and entering the analysis parameters into the parameter boxes. Second, directly specifying analysis parameters in a URL link to the ODA website (examples are given in the next section). Finally, through an ODA application programming interface (API), oda_api⁵, for instance, from a Jupyter notebook, also on their local computer.

Fig. 1. Architecture of the INTEGRAL ODA system.

The full process can be schematised as follows:

First, the request can be sent using the front-end or the oda_api client. Both interfaces verify the syntactical correctness and completeness of user queries. Second, requests arrive at the dispatcher and are processed there by an internal abstraction process, which implements classes (interfaces) for specific instrument data products, such as spectra or light-curve data, and post-processing products, such as mosaic images, light-curve images, and spectral fits. Finally, each data product interface communicates with a specific back-end, that is, data server(s) implemented as Docker⁶ containers running OSA in service mode. The containers are currently deployed locally on the HPC resources of the University of Geneva, but they could be deployed on any other HPC or CC services.

The request can either be synchronous or asynchronous. In the latter case, a continuous report from the back-end brings information to the dispatcher regarding the process status until the product is ready. In the former case, a single report is provided. Data products provided by the data server upon analysis requests are stored in a data repository and are made available to the dispatcher.

In the current version of ODA, the dispatcher also performs post-processing and visualisation of the data using specific services. This provides high-level products to be displayed on the front-end. Final products are available to the user either through the front-end or through the client API. The front-end displays sky images, spectra, light curves, and source catalogues in a virtual-desktop environment embedded in the web browser, providing the possibility of downloading the data products to the local user computer.

2.1. Interactions of the front-end interface, dispatcher, and data server

IBIS and JEM-X are coded-mask instruments that rely on a dithering pointing strategy with individual exposures called science windows (ScW) lasting 0.1–1 h. The reconstruction of sky images creates a catalogue of detected sources. This catalogue should be used during the extraction of spectra and light curves of specific sources. Separating signals from different sources in the field of view requires a sky model that lists the positions of all possible sources (see Goldwurm et al. 2003, for details). The default ODA workflow allows the user to select the data set, obtain a catalogue of detected sources by reconstructing an image, and then manipulate this catalogue to extract spectra and light curves.

The data processing is initiated in response to the user queries defining the analysis parameters (Fig. 2). These queries include at least the

• source name or sky coordinates, and

• the time interval for the data or ScW list.

Fig. 2. Left: general parameter space of ODA front-end. Right: example of the instrument-specific parameter field for the ISGRI telescope.

The front-end is able to resolve astronomical source names by accessing two of the main astronomical databases (SIMBAD and NED). It accepts time parameters in several different conventional formats. The list of ScWs can be specified as a comma-separated list of their unique identifiers. The ScW data base is separately accessible through the w3browse interface on the web pages⁷.

In addition to these generic query parameters, the front-end allows the user to specify parameters that are specific to the INTEGRAL instruments ISGRI, JEM-X, and SPI ACS (SPI-ACS). An example of the parameter field for ISGRI is shown in the right panel of Fig. 2. For ISGRI and JEM-X, it is possible to specify

• one of the two currently used versions of OSA: 10.2 and 11.0;

• the radius of the region of interest within which pointing data are selected (which depends on the instrument field of view);

• one of the two units of the JEM-X instrument (JEM-X1 or JEM-X2);

• the type of the data product (image, spectrum, or light curve);

• the energy range of the image and light curve. It should be noted that the spectrum is always computed in the full energy range with predefined resolution (16 channels for JEM-X, 128 channels for ISGRI;

• the minimum detection significance of sources in the output catalogue from imaging;

• the time binning for the light curve;

• the source catalogue to be used in the spectral and timing analyses.

In a similar way, the parameters can also be specified in the API requests using the oda_api Python package. For example, the OSA version can be specified by setting a parameter osa_version=‘OSA10.2’ in the API requests⁸.

The front-end provides a display of the high-level data products (images, spectra, light curves, source catalogues) through a virtual desktop environment. It also provides the possibility of performing a post-analysis of the data, such as fitting spectra with XSPEC spectral models, using public domain rendering packages⁹.

2.2. Data analysis and storage organisation

The ODA infrastructure uses the online archive of INTEGRAL raw data provided by the ISDC¹⁰. The task of the data server is to provide high-level data products corresponding to the user requests received through the dispatcher. Running OSA is time consuming (about 50 central processing unit (CPU) hours for a spectrum in a typical short transient observation, or about 2000 CPU hours for an analysis of historic data for a typical source), therefore it is desirable to keep pre-computed products for future (re-)uses as far as possible. However, it is not possible to store high-level data products for all imaginable combinations of user input parameters. In ODA, the permanent archive of the raw data is complemented by an analysis cache containing high- and intermediate-level data products that are added or removed depending on the the user demands averaged over certain time periods.

The cache storage is organised according to the data lineage (e.g., Ikeda & Widom 2009), which is a specific case of data provenance (Gupta 2009). The data lineage metadata comprise the information on the sequence of analysis steps (the analysis or workflow nodes) undertaken to produce the high- or intermediate-level data products. The ontology of the workflow nodes prescribes specific metadata associated with each step and induces the collection of metadata of the final product.

The lineage metadata of the cache storage contains all relevant properties of all stored data products, and only this information. This provides the possibility of reusing previously produced intermediate-stage data products for the processing of new data analysis requests by users. New data analysis workflows typically do not start processing of raw data from scratch. Instead, they are formed from a combination of parts of already available workflows derived from specific intermediate- or high-level data products stored in the cache, together with the provenance directed acyclic graph (DAG) metadata. This approach provides an efficient way to speed up the data processing following user requests if these are repetitive or recursive, or if the requests are nearly identical to those of previous users with only moderate parameter modifications.

Efficient reuse of parts of the OSA-based data analysis workflow is enabled by the reorganisation of OSA in data analysis units expressed as Python classes, following the declarative data analysis (DDA) approach inspired by the principles of functional programming. This development was driven by the needs of efficiently managing the data of INTEGRAL together with the information on the 17-year history of the telescope operations: by 2018, there were about 10³ different types of data in the raw-data archive, occupying 20 terabyte (TB) in some 2 × 10⁴ files.

The refactored OSA implementing the DDA approach (called data-driven OSA (DDOSA)¹¹) follows a simple logical scheme suitable for reproducibility of the analysis. Each analysis unit is a pure function of its input data, meaning that it only depends on its own explicit input. It transforms the input data into other data products. Any data are uniquely identified by a tree of connected analysis units that were used to produce it, or equivalently, by its DAG provenance graph. In other words, DDOSA uses provenance as a data identifier (see Savchenko 2020, for more details).

The high-level data products associated with very long analysis chains may eventually be associated with very large provenance graphs. An example of the provenance graph for a single ScW image high-level data product is shown in Fig. 3.

Fig. 3. Example of high-level provenance graph for a sky image derived from a single INTEGRAL/ISGRI ScW.

The DAG provenance graph approach for data identification at different analysis levels is optimal not only for caching frequently reused intermediate analysis step results, but also for the parallelisation of the analysis. The DAG structure of DDOSA workflows implies the presence of different independent branches of analysis that can naturally be executed independently in a distributed environment. This is taken into account in the system of job scheduling. For each analysis unit, execution requests originate either from the users (via dispatcher) or from other analysis units. Each request processing starts from the evaluation of the request, resulting either in the retrieval of the requested products from the storage cache or in the delegation of the request to a local or remote executor, a process that is transparent from the point of view of the request. A simple scheduling layer has been implemented following this approach. The advantage of this scheduler is the straightforward treatment of complex dependencies.

3. Benchmark analysis results

The web-based ODA interface retains all the functionalities of OSA and can be used to obtain publication-quality results of an analysis of INTEGRAL observations with no difference from what an experienced user can obtain running OSA locally. In this section, we demonstrate the performance of the ODA-based analysis on benchmark cases by showing that the results obtainable with ODA are compatible with previously published results or improve on them, owing to upgraded algorithms and calibration.

3.1. Crab pulsar and nebula

The Crab pulsar and nebula complex is one of the brightest sources on the X-ray sky (Hester 2008). Because of this property and its flux stability, it is often used as a calibration source in high-energy astronomy. INTEGRAL observations of Crab are reported in a number of publications (e.g., Mineo et al. 2006) and in particular, in the context of the study of the long-term variability of the source emission (Wilson-Hodge et al. 2011). We verified the performance of ODA by reproducing the published results on Crab variability and extending the evolution study over a 15-year time period. The magnitude and details of the Crab variability as observed by ISGRI are not identical to those reported by other instruments. The excess variability is in part due to the systematic uncertainties in the ISGRI calibration and in part due to the difference in the instrument energy ranges. A detailed evaluation of an OSA reconstruction of ISGRI observations as well as a discussion of the ISGRI calibration challenges is beyond the scope of this work. Here we demonstrate that ODA reproduces the best available OSA results. ODA will naturally follow any upgrades of the OSA software and calibration files in the future.

The ODA interface is currently limited to single requests for data analysis based on no more than 50 ScW to limit the waiting time on the available resources (see Sect. 4 for details, future plans, and a work-around). If the requested time span of data extends over more than 50 ScWs, a random selection of ScWs within the specified time limits is performed. This is the case for the results reported in Figs. 4, 5, and 6. The time interval of the analysis for this specific ScW subset is 2003 March 3, 23:27:40.0 to 2018 March 16, 00:03:15.0 UTC, spanning over more than 15 years. Pointings within 15 degrees from the Crab position are selected.

Fig. 4. Mosaic image of the Crab region extracted from a sample of 50 randomly selected ScWs, together with the display of the catalogue of detected sources. The result can be re-generated using the URL https://doi.org/10.5281/zenodo.3634648.

Fig. 5. Display of the Crab unfolded spectra extracted during each year from 2003 to 2018 from samples of 50 randomly selected ScWs. The fitted model (in black) is a double-broken power law with breaks fixed at 20 and 100 keV. The analysis results can be regenerated through oda-hub notebook Fit-Crab-Spectra.ipynb run after Crab_spectra_lc.ipynb.

Fig. 6. Crab long-term (15-year time span) light curve extracted from a sample of 50 randomly selected ScWs in 30–100 keV band using OSA10.2. The light curve is generated via Crab_spectra_lc.ipynb via oda-hub.

Our example Crab image could be accessed or regenerated by directly relaunching the analysis via an URL¹² in which the analysis parameters are specified after the sign +?+ and separated by the +&+ sign, for example: src_name=Crab&RA=83.633080&Dec=22.014500&radius=15 for the source name and/or sky coordinates and region of interest specification, &T1=2003-03-15T23:27:40.0 &T2=2018-03-16T00:03:15.0 &T_format=isot for the time interval, &E1_keV=20 &E2_keV=40 for the energy interval, &instrument=isgri &osa_version=OSA10.2 &product_type=isgri_image for the instrument, software version, and data product specification (spaces are added for readability but should be removed in the actual query). The parameter detection_threshold=7.5 results in a display of the sources detected at a significance level higher than 7.5. The analysis with specified parameters is launched automatically as soon as the instrument parameter is defined: &instrument=isgri. The parameters that are not explicitly specified in the parameter field of the URL are fixed to their default values.

The executable URL with all specified parameters for each data product could be obtained by pressing the ‘Share’ button displayed in the data product window on ODA front-end (see Fig. 4).

This example analysis could also be launched from a Python interface on the user laptop (e.g., from a shell or a Jupyter notebook) by providing parameters to the request

from oda_api.api import DispatcherAPI
disp=DispatcherAPI(
        host=
'www.astro.unige.ch/cdci/astrooda/dispatch-data')
disp.get_product(
        RA=83.633080,
        Dec=22.014500,
        radius=15,
        T1='2003-03-15T23:27:40.0',
        T2='2018-03-16T00:03:15.0',
        E1_keV=20.0,
        E2_keV=40.0,
        instrument='isgri',
        product='isgri_image',
        osa_version='OSA10.2')

The API code for each data product can be obtained directly by pressing the ‘API code’ button in the product window on the ODA front-end (see Fig. 4).

A crucial part of the imaging analysis is the search for significantly detected sources both in individual ScWs and in the mosaic image. Setting the source detection threshold to 7.5σ (a parameter in the web form of ODA) results in our example in the detection of four sources displayed in the image in Fig. 4. Details of the source detection are available in the catalogue display that can be accessed through a button from the image display panel, as shown in Fig. 4. Occasionally, sources may have multiple appearances in the catalogue display because this table combines several output catalogues of the standard INTEGRAL analysis, namely results of the search of sources in the mosaic and in individual ScWs. This might be important because some flaring sources are detectable in individual ScWs during short time periods, but are not detectable in mosaic images with longer exposure times (as is typical for bursting and transient sources, see e.g., Mereghetti et al. 2020 for a recent example). The user is asked to carefully inspect the output catalogue from the imaging step and adjust the source selection for the following spectral and timing analyses.

Imaging, spectral extraction, and timing routines of OSA use catalogues of sources to match the shadow patterns corresponding to these sources on the detector plane. The catalogue used for imaging and spectral or timing analysis could be explicitly specified in the ‘User catalog’ parameter window in the parameter panel. If no user catalogue is specified, the default general INTEGRAL catalogue is used. This is advisable for the imaging products but sub-optimal for the extraction of spectra and light curves, which relies on fitting a sky model on the shadow-gram. If this sky model is redundant, the fitting becomes more problematic, resulting in unreliable flux determinations. The user can edit the catalogue entries in the display of the catalogue output of the imaging step. This display also has a ‘Use catalog’ button, which would push the edited catalogue to the ‘User catalog’ to be used at the subsequent stages of analysis. The catalogue can also be defined explicitly in the form of a python ‘dictionary’ in the URL parameter field. The correctly formatted catalogue embedded in the URL can be obtained by clicking the ‘Share’ button next to the displayed data product.

The display of results of spectral analysis for all sources listed in the catalogue (user custom catalogue or the output catalogue from the imaging step analysis) provides the possibility of choosing a spectral model to fit the spectrum¹³. The display of the spectrum together with the fitted model also provides the details of the fit, such as the model parameter values with their uncertainties. The spectra are only binned at the plotting stage. The fit is performed on the spectrum at full resolution, which can be downloaded from the web interface. We note that the 20–30 keV band is affected by the long-term evolution of the ISGRI response as the ISGRI energy threshold is gradually increasing with time and low-energy events are lost. For consistency, only data above 30 keV are thus automatically fitted in the web interface, but data at lower energy are available upon download of the FITS-format spectral file¹⁴.

Figure 6 shows the 30–100 keV light curve of the source during a 17-year time span, extracted from the same set of 50 random ScWs and binned into ScW-long time bins. The figure shows the fit of the light curve with constant and linearly changing flux models. There is a noticeable decrease of the source count rate toward later times, which becomes especially pronounced after MJD 56800 (mid-2014). Superimposed onto this instrumental trend is the true variability of the Crab nebula studied by Wilson-Hodge et al. (2011).

This rapid decrease of the count rate is due to the decrease in instrument response at low energy and is not corrected for in version 10.2 of OSA because calibration algorithms were not able to correct this rapid evolution and calibration files were frozen at this moment. The correct instrument response after MJD 56800 is provided by version 11.0 of OSA with the relative calibration files¹⁵. See Sect. 4 for details on the instrumental effects contributing to these results.

Figure 7 shows a comparison of the long-term variability of the Crab flux in the 30–100 and 100–300 keV ranges for 17 years of INTEGRAL operations together with those measured with the Swift/BAT and Fermi/GBM telescopes (Wilson-Hodge et al. 2011). To produce this figure, we selected random sets of 50 ScWs spanning one-year time intervals and extracted ScW-by-ScW light curves by specifying 10 ks time steps in the ODA parameters for light-curve time binning (this is longer than the duration of one ScW). These light curves are subsequently averaged into the time bins used by Wilson-Hodge et al. (2011) for comparison. For this workflow, we exploited the API access to ODA platform, as coded in the Jupyter notebook¹⁶, which is part of the GitHub repository¹⁷. In Sect. 3.4, we detail how the online Jupyter notebooks are run as the one used to produce Fig. 7.

Fig. 7. Evolution of the Crab count rate over 17 years of INTEGRAL operations in 30–100 keV (top panel) and 100–300 keV (bottom panel) energy ranges compared to those observed by Swift/BAT (magenta) and Fermi/GBM (green data points; Wilson-Hodge et al. 2011). Grey data points show the light curve binned in ScW by ScW time bins by the ODA. Blue data points show the rebinned individual ScW measurements. The Jupyter notebook Crab_lc_longterm.ipynb can be launched used to generate these long-term light curves via API access to ODA.

Figure 7 shows that the light curves extracted using OSA10.2 and OSA11.0 in their time intervals of validity are compatible with the light curves of Swift/BAT and Fermi/GBM after they have been normalised to their average level. The intrinsic variability of the Crab nebula (at ∼5% level) can be appreciated in the general trend of all instruments, as was done by Wilson-Hodge et al. (2011). There are some residual differences between the INTEGRAL and Swift/BAT or Fermi/GBM light curves, however, which can be used to estimate the systematic cross-calibration uncertainty. This can reach up to 10% in the 100–300 keV energy range. The count rate light curve has variations that are also due to the evolution of the instrument gain; these are accounted for using response files to recover the intrinsic source flux in the spectral extraction stage.

The ODA interface allows us to also extract images, spectra, and light curves of JEM-X instrument using the same set of parameters as for ISGRI. Figure 8 shows the 3–20 keV image of the Crab extracted from 50 randomly selected ScWs with pointing directions within 5° from the Crab (to ensure that the source is in the field of view of JEM-X.

Fig. 8. Image of the sky region around Crab obtained with the JEM-X1 instrument in the energy range 3–20 keV. The image can be regenerated via URL https://doi.org/10.5281/zenodo.3832080.

The Crab is detected with a significance in excess of 10³σ in the image. Two sources are detected with a significance higher than 20 in the image: the Crab, and the X-ray binary 1A 0535+262. We used a catalogue containing these two sources for the spectral extraction.

Figure 5 shows combined ISGRI + JEM-X1 unfolded spectra of the spectrum of the Crab, extracted from the 50 ScW data sets on year-by-year basis from 2003 to 2018. We modelled the spectra with a broken power law with two break energies fixed at 20 and 100 keV. The first value is meant to catch possible differences between the JEM-X1 and ISGRI spectral responses; the second is a reasonable approximation for the most probable spectral shape of the Crab (Jourdain & Roques 2009). We applied a systematic factor of 2% to all spectral data and limited the JEM-X1 data between 5 and 20 keV and the ISGRI data between 20 and 300 keV. The number of degrees of freedom is 55 for OSA10.2 (before 2016) and 112 after 2016 (OSA11.0). Figure 9 shows the year-by-year change of the Crab spectrum measurements: Our results are roughly in line with previous findings (e.g., Mineo et al. 2006). Formally not acceptable fits are present in 2003 and 2005 with an anomalous power-law spectral index Γ₂ (20–100 keV) due to a change of the low threshold in the ISGRI detector, which is currently taken into account in a sub-optimal way by the calibration files. Similarly, an anomalous low flux in 2017 for ISGRI is due to calibration issues that are currently investigated by part of our team in a parallel effort.

Fig. 9. Best-fit spectral parameters with uncertainty intervals at the 68% confidence level on a single parameter and the reduced χ2 fit statistics of the Crab pulsar and nebula ISGRI + JEM-X1 spectra averaged on a year-by-year basis; the first point refers to 2003 and the last point to 2018. The data extend from 3.5 to 20 keV for JEM-X and 20–300 keV for ISGRI. The model is a broken power law with break energies fixed at 20 and 100 keV; Γ1−3 is the spectral index at increasing energies. We allowed the spectra to assume different normalisations through the flux parameter that we report for 5–20 keV (JEM-X1) and 20–100 keV (ISGRI). The resulting number of degrees of freedom is 55 before 2016 and 112 after 2016.

3.2. Extremely bright source: V404 Cygni

V404 Cygni is a microquasar that underwent a spectacular outburst in 2015 during which the source flux reached 50 Crab (Sánchez-Fernández et al. 2017; Rodriguez et al. 2015). A high flux like this might pose challenges for a data analysis because of saturation or pileup effects that need to be properly taken into account. To validate the performance of ODA for the bright source case, we reproduced the results of the analysis of V404 Cyg reported by Rodriguez et al. (2015). At the first stage, we assessed the overall evolution of the source throughout the activity period that lasted from 2015 June 20 T15:50:00.0 UTC to 2015 June 25 T04:05:59.0 UTC, analysed by Rodriguez et al. (2015). Entering this time interval into the ODA interface via the URL, we let the system select 50 ScWs at random for the analysis with pointing directions within 10 degrees from the source direction and produce a mosaic image shown in Fig. 10. The source is the brightest source in the image and is detected with significance exceeding 1000σ. The next brightest source in the field is Cyg X-1. A very strong source produces ‘ghost’ images due to the specific of the coded-mask imaging technique. This is readily seen in the example of V404 Cyg by extracting all the sources in the image that are detectable at a significance threshold above 5σ: This would result in very large number of ‘ghosts’ that would appear in the resulting catalogue as ‘NEW’ sources.

Fig. 10. Significance map of the V404 Cygni region in the 20–40 keV band. The image can be regenerated via URL https://doi.org/10.5281/zenodo.3634669.

Figure 11 shows the JEM-X1 and ISGRI light curves of V 404 Cyg extracted for this period. Given the problem of detection of ghost sources around a very bright source, it is important to correctly define an input source catalogue for the light-curve production. Otherwise, the catalogue that would be produced based on the imaging step of the analysis would include all the ghost sources in the procedure of fitting the detector image, which would lead to either incorrect flux calculations or larger error estimates compared to the flux measurements including only real sources.

Fig. 11. JEM-X1 (top) and ISGRI (bottom) light curves of V404 Cygni during the 2015 flaring period (red data points). Black lines are from Rodriguez et al. (2015).

Figure 11 shows that the source underwent several short-time flares. The overall evolution of the source flux inferred from ODA is practically indistinguishable from that reported by Rodriguez et al. (2015). This is clear from direct comparison of the two results, shown in Fig. 11, where black shows the light curve of Rodriguez et al. (2015) and red shows the result extracted using ODA.

3.3. Crowded field: GX 5−1

To verify the performance of ODA in the crowded field, we considered an example of GX 5−1, a persistent bright low-mass X-ray binary with a neutron star. It is located in the inner Galaxy region in a crowded field with many bright sources. In this situation, OSA has to simultaneously take into account all the bright sources while modelling the superposition of the shadows of different sources on the detector. If this is not done properly, the signal of the source of interest could be contaminated by the overlapping shadows of unaccounted-for sources. Moreover, if bright sources are not included in the sky model for spectrum or light-curve extraction, their photons will be erroneously assigned to other sources or background, which hampers a reliable estimate. However, neglecting weak sources is a minor concern when the interest is in bright sources because contamination can reach a few percent at most of the contaminant’s flux in the worst cases.

To provide a direct comparison with the published results by Paizis et al. (2005), we reproduced their selection of 44 science windows when the source was observed by the Galactic Plane Scan (GPS) and Deep exposure of the Galactic Central Radian (GCDE) programs in 2013 with a maximum offset of three degrees. We first made an image in the 20–30 keV band (Fig. 12) to select sources with a minimum significance of 7 for the subsequent spectral extraction. This resulted in a catalogue of 25 sources. Owing to the construction of OSA, some sources might appear multiple times in the catalogue, therefore it is necessary to delete duplicates from the catalogue that is used as input in the following steps.

Fig. 12. Zoom of the significance map of the sky region around GX 5−1 in the 20–30 keV energy band obtained from a selection of 44 science windows belonging to the Galactic Plane Scan and Deep exposure of the Galactic Central Radian programs carried out in 2013 with a pointing offset of less than three degrees from GX 5−1. This image can be obtained from the following URL.

We performed the spectral extraction of both JEM-X2 and ISGRI data using the Python API with the same selection of science windows. The dead-time and vignetting-corrected exposures for JEM-X2 and ISGRI are 40.6, and 57.2 ks, respectively. We added 1% flux systematic to ISGRI and 3% to JEM-X2, and we ignored JEM-X2 data below 3 and above 20 keV and ISGRI data below 20 and above 70 keV owing to the particular spectral shape. We modelled the joint spectrum with the same spectral models as Paizis et al. (2005): the ‘western’ model consisting of a compTT+bbodyrad, and the ‘eastern’ model, consisting of compBB+diskbb using XSPEC. We also introduced a cross-normalisation factor fixed to one for ISGRI to account for the non-simultaneity of some data (see Fig. 13).

Fig. 13. Time-averaged spectrum of GX 5–1 extracted from the same 44 science windows as in Paizis et al. (2005), collected in 2003 with a maximum pointing offset of 3° from the source. Red stars and black circles represent JEM-X2 and ISGRI data, respectively. The dotted lines represent the two components of the eastern model with best-fit parameters (Table 1).

We determined the 1σ confidence ranges of the parameters using a Monte Carlo Markov chain with the Goodman-Weare algorithm (Goodman & Weare 2010) as implemented in Xspec v. 12.11.0, and taking the 16, 50, and 84% percentiles of the posteriors as lower, central, and upper values. For the chain, we used 40 walkers, a length of 26 000, and a burn-in phase length of 6000. The results are presented in Table 1: The two models represent the spectra well. However, the evolution of the instrument calibration and analysis algorithms has led to significant differences in the parameter values between the 2005 work, which was made with OSA v4.2, and our work, which was performed with OSA v.10.2. This is inherent to OSA and not directly related to the interface used to extract these spectra. A Python notebook with the full workflow is available online¹⁸.

3.4. 3C 279

In spite of its high bolometric luminosity and powerful jet (Hayashida et al. 2015), the active galactic nucleus (AGN) 3C 279 is a weak source for INTEGRAL. It has a hard spectrum in the hard X-ray energy band. Its flux is at the level of the sensitivity limit of ISGRI, and its detectability depends on the source state. One of the flaring episodes occurred in 2015, and INTEGRAL observations of 3C 279 during this episode were obtained as a target-of-opportunity campaign. The results of the data analysis for this TOO are described by Bottacini et al. (2016).

Following the same approach as for other sources, we first performed an exploration of the source behaviour throughout the 15-year time span of the data. However, this source would likely be only marginally detected in any 50 ScW exposure, and no assessment of the variability pattern is possible in this way.

Instead, the full data set has to be explored to determine the periods during which the sources are detected. We used the API access to ODA to work with data sets that are longer than 50 ScWs in the following way. As for the Crab light-curve case in Sect. 3.1, the Jupyter notebooks for the 3C 279 analysis can be launched¹⁹

GitHub repository, which is integrated with the Binder interactive notebook service²⁰. Launching the binder using the ‘launch binder’²¹ button in the repository²² and choosing the notebooks for the 3C 279 light curve and spectra found in examples makes it possible to generate the results online that are described below.

At the first stage of the analysis, we determined the bright sources in the source field. We generated a mosaic image of the field and used the output catalogue of the mosaic analysis, adding 3C 279 explicitly to the catalogue as an input step for the timing and spectral analysis. Using the resulting source catalogue, we processed all sets of 50 ScWs in sequence to obtain the long-term light curve of the source shown in Fig. 14. This figure shows that the source is systematically detected throughout the entire 15-year time span. It shows moderate (if any) variability from year to year.

Fig. 14. Light curve of 3C 279 in the 15 yr time span. Grey vertical lines show exposure periods of the source. The notebook 3C279_lc.ipynb for the calculation of the light curve can be launched using this URL.

The 2015 flare of the source reported by Bottacini et al. (2016) can be identified as the highest flux point in the light curve in Fig. 14. A more detailed view of the light curve for the flaring episode discussed by Bottacini et al. (2016) is shown in Fig. 15, where we plot (red points) the ODA API light curve, extracted for the full range of the flaring period investigated in Bottacini et al. (2016), and the ISGRI light curve reported in Fig. 1 of Bottacini et al. (2016). The average flux of the Bottacini et al. (2016) light curve is compatible with our bin; it overlaps the same time span. The average count rate is at the level of 1 ct s⁻¹, which agrees with the published value. This light curve can be regenerated using the same light-curve extraction notebook as for the long-term light curve of the source, changing the time interval to focus on the flaring period, July 2015, and adjusting the energy range. This shows how the notebook that is available for on-the-fly re-deployment²³ can be reused to refine or reuse the analysis for different energy ranges or different sources.

Fig. 15. Light curve of 3C 279 in 20–100 keV range for the flare period reported by Bottacini et al. (2016; red points), extracted using the ODA API, and the ISGRI light curve reported in Fig. 1 of Bottacini et al. (2016; blue points), corresponding to their ISGRI data time span. The notebook 3C279_lc_flare.ipynb to regenerate the result can be executed online at this URL, and can be re-executed via Binder integration of the oda-hub project.

Figure 16 shows a comparison of the time-averaged spectrum of the source with the flaring state spectrum. We used the same spectral range extraction of 20–100 keV and the same spectral model (flux-pegged power- law model pegpwrlw²⁴) as in Bottacini et al. (2016). The spectral fit of the flaring-state spectrum shown in Fig. 16 has a very hard slope and a flux in the 18–55 keV band of ( both reported at 90% c.l.), which are consistent with Bottacini et al. (2016), where the authors reported a photon index of and a flux of .

Fig. 16. Comparison of the time-averaged spectrum of 3C 279 (black) with the spectrum of the flaring state observed during the TOO period reported by Bottacini et al. (2016) in blue; the upper limit is at the 3σ confidence level. The respective models are represented as solid lines. The notebook 3C279_spectrum.ipynb to calculate the spectra can be launched using this URL.

3.5. NGC 2110

NGC 2110 is an example of a moderately bright Seyfert galaxy, that is, a representative of a typical hard X-ray bright and persistent AGN (Peterson 1997). AGN of this type are the most abundant extra-galactic sources observed by INTEGRAL.

Figure 17 shows the significance image of the source region extracted from a set of 50 random ScWs spread over 15 years of INTEGRAL operations. NGC 2110 is the brightest source in the field. Its detection significance reaches ≃17σ in the exposure T ≃ 90 ks. The dimmer source H 0614+091 is detected at a significance level 10σ, and no other source in the field is detected with a significance exceeding 5σ.

Fig. 17. 20–40 keV image of the NGC 2110 region extracted from a set of 50 randomly selected ScWs. The image can be regenerated via the URL https://doi.org/10.5281/zenodo.3634584.

Figure 18 shows the long-term light-curve of the source extracted from the sequence of 50 ScW data-sets pointed within 10 degrees from the source, using the same notebook as for the 3C 279 analysis, part of the repository²⁵. The statistical uncertainty of the flux measurement in individual ScWs is too large and it is not possible to follow the long-term evolution of the source spectrum with ScW-by-Scw binning. Taking this into account, we have rebinned the light-curve into wider time bins. Several variability episodes are clearly identifiable with such binning in the long-term light-curve.

Fig. 18. Rebinned light curve of NGC 2110 extracted from the sequence of 50 ScW-long intervals using the notebook lc_longterm.ipynb. The result can be regenerated via notebook deployment on Binder at this URL.

We used the same approach as for 3C 279 to extract the source spectrum with a long exposure time. We extracted the spectrum of NGC 2110 by stacking the spectra in sequences of 50 ScW-long exposures and calculating the weighted average using the same notebook as for the extraction of the 3C 279 spectrum, which is available and executable at the project page²⁶. The resulting source spectrum with a total exposure of 2.4 Ms is shown in Fig. 19. The figure shows two separate spectra for the periods of applicability of OSA 10.2 and OSA 11.0. The physical origin of the hard X-ray emission from Seyfert galaxies is thought to be due to the inverse Compton scattering of emission from an accretion disk in a hot corona characterised by certain temperature (Lubiński et al. 2016). The suppression of the emission above 100 keV expected in this model is seen in the longer (1.8 Ms) exposure spectrum for the OSA 10.2 applicability period. However, the exposure of the OSA 11.0 period (0.6 Ms) is not sufficiently long to constrain a high-energy cut-off in the spectrum. In order to compare our results with those reported in Lubiński et al. (2016), we extracted ScWs for the same time span and angular extraction cone radius. We have a final exposure of ≃1714 ks. We stress that a full reproduction of the analysis reported in Lubiński et al. (2016) is beyond the scope of our work because we did not use soft X-ray data to constrain the parameters of the COMPPS Xspec model that was used to describe inverse Compton emission. In our analysis we used the same model as in Lubiński et al. (2016), and we fixed all the parameters to those reported in their analysis, except for the electron temperature kT_e, the y-Compton parameter, and the normalisation. In detail, we fixed the seed photons temperature T_bb to a value of 10 eV, the amount of reflection R to 0.63, and the geometry to the spherical case. We estimated the 90% parameter confidence range using the Xspec (v. 12.11.0) MCMC implementation based on the Goodman-Weare algorithm (Goodman & Weare 2010) with 20 walkers, and a Cauchy proposal distribution, running a 10 000-steps chain, with a burn-in phase of 3000 steps. We ran the chain with an initial state starting from the Xspec best-fit solution. We report the 0.05, 0.5, and 0.95 quantiles of the posterior distribution as lower, central, and upper values. We find a temperature keV, a normalisation constant , and a value of the y-Compton parameter of , compared to the values reported in Lubiński et al. (2016) of kT_e = 2.3 ± 0.5 × 10² keV, , and , respectively. The spectrum and the frequentist best-fit model are reported in the top panel of Fig. 19.

Fig. 19. Top: spectrum of NGC 2110 extracted from the stacking of spectra obtained for 50 ScW sets in the time periods used in Lubiński et al. (2016) using OSA 10.2. Bottom: comparison of the spectra of NGC 2110 extracted from the stacking of spectra obtained for 50 ScW sets in the time periods used in Ursini et al. (2019) and the spectra published in Ursini et al. (2019). The spectra can be regenerated online using the notebook spectrum_longterm.ipynb and spectrum_longterm_Ursini19.ipynb.

As a further benchmark, we compared the results from ODA to those published in Ursini et al. (2019). In this case, the authors distributed the INTEGRAL/ISGRI spectra online that were extracted with OSA and that they used for their publication. We extracted the INTEGRAL/ISGRI spectra with ODA for the same time span and spectral window as in Ursini et al. (2019). The results are shown in the lower panel of Fig. 19.

4. Analysis limitations

In the current ODA implementation, the dispatcher defines which scientific analysis workflows can be executed through the front-end, and enables two versions of the INTEGRAL analysis software: Standard OSA10.2 and OSA11.0. OSA10.2 can only be used for the analysis of the data taken before 2016. OSA11.0 can only be used for the ISGRI data since 2016 (at the time of writing), while it can be used for JEM-X during the full mission lifetime. This will change as soon as updated ISGRI calibration files are made available. It should be remarked that the introduction of new releases of OSA will be made available.

The maximum number of ScW allowed in a single request is set to 50 (corresponding to up to 50 CPU-hours of computing) to reduce load on the system through too long jobs. This might change if the back-end is deployed in a different location or more computing resources are made available. Large requests will also be available for authenticated users. It is currently possible to overcome this limitation by looping over successive sets of 50 ScWs, as we demonstrated in the analysis examples in this paper.

The INTEGRAL data analysis within the ODA platform entirely relies on scientific data analysis with OSA, modifying only the mechanism with which individual components of OSA are invoked (principally to allow for preservation and reuse of intermediate analysis results, as explained above). This means that the INTEGRAL analysis within ODA shares the limitations of OSA, and any future changes in OSA will be available in ODA. A detailed scientific validation of the long-term evolution of the INTEGRAL instruments and the current status of the data analysis software with OSA goes beyond the scope of this paper.

5. Comparison with other online analysis platforms

Several other online data analysis platforms offer similar services. Some of them offer them for INTEGRAL instruments.

The high energy astrophysics science archive research center (HEASARC) Hera exposes much of the high energy astrophysics software (HEASoft) capabilities (with no relevant support for INTEGRAL) as a web service. This original and promising development has limitations of the service resources. Hera was in part an inspiration for the most superficial scheme of INTEGRAL ODA back-end.

The Swift/X-Ray Telescope (XRT) online data analysis²⁷ is a particularly successful example of an online analysis and has encouraged adoption of this strategy for other instruments. The XMM-Newton remote interface for science analysis (RISA)²⁸ is conceptually similar to the Swift/XRT online analysis, but is arguably less well known, perhaps because the strategy of observation scheduling in XMM-Newton is different: While Swift favours a large number of short public observations with quick impact, XMM-Newton observations are often part of larger campaigns and private observations, favouring a traditional offline analysis.

The Swift/XRT and XMM-Newton telescopes both feature focusing mirrors, and their data analysis workflows require much smaller computing resources than INTEGRAL. Coded-mask telescopes feature a much larger FoV than the focusing telescopes, which in turn results in a larger number of sources to be considered by the decoding process. In addition, their point spread function (PSF) is highly non-trivial and generally spread over the entire FoV. The combination of the typically long observations and dependence on a model with a potentially large number of sources means that an online analysis of coded-mask telescopes is resource-hungry and has to adopt a very different approach to handling data analysis workflows and pre-computed data.

The INTEGRAL SPI data analysis interface (SPIDAI) provides an online data analysis for INTEGRAL/SPI. The SPI data analysis requires considerable re-analysis for each new request. This is in principle similar to the ODA ISGRI and JEM-X workflows, but without the comparably extensive reuse of pre-computed results. A SPIDAI analysis can avoid this additional complexity because only a relatively small number of sources are sufficiently bright to be observed by SPI.

The high-energy astrophysics virtually enlightned sky (HEAVENS)²⁹ follows a very different approach from that of INTEGRAL ODA, SPIDAI, Swift/XRT, RISA, or Hera. It pre-computes the raw data using undisclosed customised procedures, creating a space-energy-time map of celestial flux with a fixed pre-defined resolution in each dimension. This considerably speeds up certain types of analyses, but also implies that the scientific meaning of the HEAVENS output may be very different from that provided by the expert-validated OSA. Furthermore, a particular pre-defined selection of space-time-energy resolutions restricts certain types of analyses (e.g., light curves with time bins shorter than about 3000 s, which is the INTEGRAL pointing duration). Finally, pre-computing the results is costly (both in computing time and human effort) and does not necessarily privilege popular results because it is not done in response to user needs. As a result, the pre-computed HEAVENS results are available with a long delay. Finally, HEAVENS imposes additional severe restrictions on the output results, for example, on the size of the requested image.

In contrast, the ODA INTEGRAL analysis runs an on-demand OSA analysis based exclusively on publicly available tools. Any improvements in OSA, officially validated by the instrument experts and the data centre, are immediately adopted by ODA (by providing an additional OSA version in the parameter selection).

Conversely, ODA operations expose some of the technical issues of OSA. While in the traditional offline analysis approach these issues would plague users with poorly understood error messages, an equivalent message in ODA-wrapped OSA can be directed straight to the OSA software developers, and the patch will be made available in the next OSA release (with a corresponding update to ODA).

INTEGRAL ODA provides a much larger set of results than that available in pre-computed databases (such as HEAVENS). For example, it is possible to produce light curves with short (down to 10 s) time bins and high-resolution spectra. More advanced users may upload custom source lists to use as a sky model to provide the most reliable INTEGRAL results, as recommended in the OSA cookbook. Pre-computing the results with this granularity would not be feasible in platforms like HEAVENS.

The ODA platform is built as a cloud-native solution, designed to be adaptable to any modern computing environment. It is already open-source and can readily be developed by adding new components. The platform consists of a range of independent components that follow common standards of communication and interoperability. This means that the platform may grow beyond a single deployment, for example, it can be deployed as a part of a federated infrastructure of ESA DataLabs³⁰.

The ODA platform is strongly committed to interoperability and integration with the European Open Science Cloud (EOSC)³¹, and has been validated as an EOSC service³².

6. Conclusions

We have presented the new approach for the INTEGRAL data analysis, which uses the cloud-computing technology to enable deployment of data analysis workflows for imaging and spectrum and timing analysis online through a web interface in a web browser³³, or through a dedicated API that can be used in Python notebooks that can be executed locally or remotely from the oda-hub project Github repository. This approach provides an important boost for the reproducibility of results extracted from INTEGRAL data and possibilities of sharing and reusing data analysis methods. The virtualisation of the data analysis system also provides a viable solution for the long-term preservation of the data and analysis results. This paper demonstrates how reusable astronomical data analysis workflows can be shared and embedded in publications.

Performance tests presented in this paper validated ODA for use in scientific publications making use of INTEGRAL data. The ODA results are identical to those that are obtained with OSA with parameter choices following standard recommendations of the OSA handbooks.

INTEGRAL ODA provides an on-demand analysis of any INTEGRAL ISGRI and JEM-X data, leveraging an almost complete set of OSA capabilities and yielding results that are identical to the expert-validated publicly released OSA. It is especially useful in the context of multi-wavelength and multi-messenger studies of variable astronomical sources (Mészáros et al. 2019) because it provides ready-to-use and flexible data products: images, spectra, and light curves that can be adjusted to specific details of source variability and observation periods of other instruments. It can also be used to explore the long-term behaviour of multi-messenger sources prior to their activity periods.

References

All Tables

All Figures

	Fig. 1. Architecture of the INTEGRAL ODA system.
In the text

	Fig. 2. Left: general parameter space of ODA front-end. Right: example of the instrument-specific parameter field for the ISGRI telescope.
In the text

	Fig. 3. Example of high-level provenance graph for a sky image derived from a single INTEGRAL/ISGRI ScW.
In the text

	Fig. 4. Mosaic image of the Crab region extracted from a sample of 50 randomly selected ScWs, together with the display of the catalogue of detected sources. The result can be re-generated using the URL https://doi.org/10.5281/zenodo.3634648.
In the text

	Fig. 5. Display of the Crab unfolded spectra extracted during each year from 2003 to 2018 from samples of 50 randomly selected ScWs. The fitted model (in black) is a double-broken power law with breaks fixed at 20 and 100 keV. The analysis results can be regenerated through oda-hub notebook Fit-Crab-Spectra.ipynb run after Crab_spectra_lc.ipynb.
In the text

	Fig. 6. Crab long-term (15-year time span) light curve extracted from a sample of 50 randomly selected ScWs in 30–100 keV band using OSA10.2. The light curve is generated via Crab_spectra_lc.ipynb via oda-hub.
In the text

Fig. 7. Evolution of the Crab count rate over 17 years of INTEGRAL operations in 30–100 keV (top panel) and 100–300 keV (bottom panel) energy ranges compared to those observed by Swift/BAT (magenta) and Fermi/GBM (green data points; Wilson-Hodge et al. 2011). Grey data points show the light curve binned in ScW by ScW time bins by the ODA. Blue data points show the rebinned individual ScW measurements. The Jupyter notebook Crab_lc_longterm.ipynb can be launched used to generate these long-term light curves via API access to ODA.

In the text

	Fig. 8. Image of the sky region around Crab obtained with the JEM-X1 instrument in the energy range 3–20 keV. The image can be regenerated via URL https://doi.org/10.5281/zenodo.3832080.
In the text

Fig. 9. Best-fit spectral parameters with uncertainty intervals at the 68% confidence level on a single parameter and the reduced χ2 fit statistics of the Crab pulsar and nebula ISGRI + JEM-X1 spectra averaged on a year-by-year basis; the first point refers to 2003 and the last point to 2018. The data extend from 3.5 to 20 keV for JEM-X and 20–300 keV for ISGRI. The model is a broken power law with break energies fixed at 20 and 100 keV; Γ1−3 is the spectral index at increasing energies. We allowed the spectra to assume different normalisations through the flux parameter that we report for 5–20 keV (JEM-X1) and 20–100 keV (ISGRI). The resulting number of degrees of freedom is 55 before 2016 and 112 after 2016.

In the text

	Fig. 10. Significance map of the V404 Cygni region in the 20–40 keV band. The image can be regenerated via URL https://doi.org/10.5281/zenodo.3634669.
In the text

	Fig. 11. JEM-X1 (top) and ISGRI (bottom) light curves of V404 Cygni during the 2015 flaring period (red data points). Black lines are from Rodriguez et al. (2015).
In the text

	Fig. 12. Zoom of the significance map of the sky region around GX 5−1 in the 20–30 keV energy band obtained from a selection of 44 science windows belonging to the Galactic Plane Scan and Deep exposure of the Galactic Central Radian programs carried out in 2013 with a pointing offset of less than three degrees from GX 5−1. This image can be obtained from the following URL.
In the text

	Fig. 13. Time-averaged spectrum of GX 5–1 extracted from the same 44 science windows as in Paizis et al. (2005), collected in 2003 with a maximum pointing offset of 3° from the source. Red stars and black circles represent JEM-X2 and ISGRI data, respectively. The dotted lines represent the two components of the eastern model with best-fit parameters (Table 1).
In the text

	Fig. 14. Light curve of 3C 279 in the 15 yr time span. Grey vertical lines show exposure periods of the source. The notebook 3C279_lc.ipynb for the calculation of the light curve can be launched using this URL.
In the text

Fig. 15. Light curve of 3C 279 in 20–100 keV range for the flare period reported by Bottacini et al. (2016; red points), extracted using the ODA API, and the ISGRI light curve reported in Fig. 1 of Bottacini et al. (2016; blue points), corresponding to their ISGRI data time span. The notebook 3C279_lc_flare.ipynb to regenerate the result can be executed online at this URL, and can be re-executed via Binder integration of the oda-hub project.

In the text

	Fig. 16. Comparison of the time-averaged spectrum of 3C 279 (black) with the spectrum of the flaring state observed during the TOO period reported by Bottacini et al. (2016) in blue; the upper limit is at the 3σ confidence level. The respective models are represented as solid lines. The notebook 3C279_spectrum.ipynb to calculate the spectra can be launched using this URL.
In the text

	Fig. 17. 20–40 keV image of the NGC 2110 region extracted from a set of 50 randomly selected ScWs. The image can be regenerated via the URL https://doi.org/10.5281/zenodo.3634584.
In the text

	Fig. 18. Rebinned light curve of NGC 2110 extracted from the sequence of 50 ScW-long intervals using the notebook lc_longterm.ipynb. The result can be regenerated via notebook deployment on Binder at this URL.
In the text

Fig. 19. Top: spectrum of NGC 2110 extracted from the stacking of spectra obtained for 50 ScW sets in the time periods used in Lubiński et al. (2016) using OSA 10.2. Bottom: comparison of the spectra of NGC 2110 extracted from the stacking of spectra obtained for 50 ScW sets in the time periods used in Ursini et al. (2019) and the spectra published in Ursini et al. (2019). The spectra can be regenerated online using the notebook spectrum_longterm.ipynb and spectrum_longterm_Ursini19.ipynb.

In the text