Testing strengths, limitations, and biases of current pulsar timing arrays’ detection analyses on realistic data

¹ Max Planck Institute for Gravitational Physics (Albert Einstein Institute), Leibniz Universität Hannover, Callinstrasse 38, 30167 Hannover, Germany
e-mail: serena.valtolina@aei.mpg.de
² Dipartimento di Fisica “G. Occhialini”, Università degli Studi di Milano-Bicocca, Piazza della Scienza 3, 20126 Milano, Italy
³ INFN, Sezione di Milano-Bicocca, Piazza della Scienza 3, 20126 Milano, Italy
⁴ INAF – Osservatorio Astronomico di Cagliari, Via della Scienza 5, 09047 Selargius (CA), Italy
⁵ Institute for Gravitational and Subatomic Physics (GRASP), Utrecht University, Princetonplein 1, 3584 CC Utrecht, The Netherlands
⁶ Institut für Physik und Astronomie, Universität Potsdam, Haus 28, Karl-Liebknecht-Str. 24/25, 14476 Potsdam, Germany
⁷ INAF – Osservatorio Astronomico di Brera, Via Brera 20, 20121 Milano, Italy

Received: 27 September 2023
Accepted: 20 January 2024

Abstract

State-of-the-art searches for gravitational waves (GWs) in pulsar timing array (PTA) datasets model the signal as an isotropic, Gaussian, and stationary process described by a power law. In practice, none of these properties are expected to hold for an incoherent superposition of GWs generated by a cosmic ensemble of supermassive black hole binaries (SMBHBs). This stochastic signal is usually referred to as the GW background (GWB) and is expected to be the primary signal in the PTA band. We performed a systematic investigation of the performance of current search algorithms, using a simple power-law model to characterise GW signals in realistic datasets. We used, as the baseline dataset, synthetic realisations of timing residuals mimicking the European PTA (EPTA) second data release (DR2). Thus, we included in the dataset uneven time stamps, achromatic and chromatic red noise, and multi-frequency observations. We then injected timing residuals from an ideal isotropic, Gaussian, single power-law stochastic process and from a realistic population of SMBHBs, performing a methodical investigation of the recovered signal. We found that current search models are efficient at recovering the GW signal, but several biases can be identified due to the signal-template mismatch, which we identified via probability-probability (P–P) plots and quantified using Kolmogorov-Smirnov (KS) statistics. We discuss our findings in light of the signal observed in the EPTA DR2 and corroborate its consistency with a SMBHB origin.