Massive galaxy formation caught in action at z ∼ 5 with JWST

We report the discovery of a compact group of galaxies, CGG-z5, at z ∼ 5 . 2 in the EGS field covered by the JWST / CEERS survey. CGG-z5 was selected as the highest overdensity of galaxies at z > 2 in recent JWST public surveys and it consists of six candidate members lying within a projected area of 1 . 5 ′′ × 3 ′′ (10 × 20 kpc 2 ). All group members are HST / F435W and HST / F606W dropouts while securely detected in the JWST / NIRCam bands, yielding a narrow range of robust photometric redshifts 5 . 0 < z < 5 . 3. The most massive galaxy in the group has a stellar mass log( M ∗ / M ⊙ ) ≈ 9 . 8, while the rest are low-mass satellites (log( M ∗ / M ⊙ ) ≈ 8 . 4–9.2). While several group members were already detected in the HST and IRAC bands, the low stellar masses and the compactness of the structure required the sensitivity and resolution of JWST for its identification. To assess the nature and evolutionary path of CGG-z5, we searched for similar compact structures in the E agle simulations and followed their evolution with time. We find that all the identified structures merge into a single galaxy by z = 3 and form a massive galaxy (log( M ∗ / M ⊙ ) > 11) at z ∼ 1. This implies that CGG-z5 could be a “proto-massive galaxy” captured during a short-lived phase of massive galaxy formation.


Introduction
Understanding the formation of massive galaxies (M * > 10 11 M ) is a major goal in modern astrophysics.In ΛCDM cosmology, massive galaxies are formed via hierarchical clustering, where massive dark matter haloes are formed through the continuous merging of low-mass subhaloes (e.g., Springel et al. 2005Springel et al. , 2008;;Boylan-Kolchin et al. 2009;Klypin et al. 2011;Somerville & Davé 2015).The assembly of dark matter haloes, as one of the most important processes in the formation of structure in the Universe, can be traced by merger graphs or trees, based on analytical approaches and simulations (e.g., Press & Schechter 1974;Somerville & Primack 1999;Zhang et al. 2008;Neistein & Dekel 2008;Roper et al. 2020).Advanced cosmological simulations have displayed great details on this process, showing that the most rapid growth of nearby massive galaxies occurred at z > 2 via the merging of low-mass satellites with M * 10 10 M as well as in situ gas accretion (e.g., Hopkins et al. 2009;Oser et al. 2010;Benson et al. 2012;Hirschmann et al. 2012).
However, the observational confirmation of these merging systems before coalescence is still lacking, especially for low-Marie Curie Fellow.mass systems at high redshift.This is not only due to the extreme faintness of low-mass galaxies at high redshift, but also their optical lines and stellar continuum are redshifted to infrared wavelengths that are not available with the Hubble Space Telescope (HST) and ground-based facilities.Now the situation has changed with the successful launch of the James Webb Space Telescope (JWST).With the unprecedented sensitivity, spatial resolution, and long wavelength coverage, we are right on the cusp of discovering more distant structures down to a significantly low-mass limit (e.g., Morishita et al. 2022).
In this Letter we report a low-mass and compact galaxy group CGG-z5 at z ∼ 5.2 revealed by JWST, a candidate of a merging system forming a massive galaxy.We adopt cosmology H 0 = 73 km s −1 Mpc −1 , Ω M = 0.27, and Λ 0 = 0.73 as well as a Chabrier initial mass function (Chabrier 2003).

JWST+HST data processing and measurements
We use the reduced and calibrated JWST+HST image products, and photometric and photo-z catalogs that are publicly L11, page 1 of 8 available1 .The JWST data reduction and calibration follow the same pipeline applied for data in Naidu et al. (2022), Bradley et al. (2022), Barrufet et al. (2022), Fujimoto et al. (2022), and Bezanson et al. (2022).A detailed description of the pipeline will be provided in Brammer et al. (in prep.).Briefly, we retrieved the pipeline-calibrated level-2 NIRCam products from the Mikulski Archive for Space Telescopes (MAST), then further calibrated the data and processed them as mosaics using the Grizli package (Brammer & Matharu 2021).The Grizli pipeline masks various artifacts and models wisps, calibrates for a flat-field, subtracts large-scale sky background, and aligns the fully calibrated images to stars from the Gaia DR3 catalog (Gaia Collaboration 2023).The NIRCam photometric zeropoint correction was applied and the derived photometric zeropoints are consistent with those derived by other JWST Early Release Science (ERS) teams (Boyer et al. 2022;Nardiello et al. 2022).The NIRCam images of short wavelength channels were processed to a final pixel scale of 0.02 , and 0.04 per pixel for long wavelength channels.Sources were first extracted in the longest wavelength image using Source Extraction and Photometry (SEP, Barbary 2016), and photometry was measured within an aperture of 0.5 diameter on the position from the first extraction.The measurement was performed in all JWST images with aperture correction.The HST images in these fields were also processed with Grizli and the photometry was measured in an identical way.The photometric redshifts were computed by fitting all available JWST+HST photometry with EAZY (Brammer et al. 2008).The catalog in the Cosmic Evolution Early Release Science Survey (CEERS) field (Finkelstein et al. 2022) consists of 14 band photometry of which seven are from JWST (F115W, F150W, F200W, F277W, F356W, F410W, and F444W), and the remaining seven are from HST (F105W, F125W, F140W, F160W, F435W, F606W, and F814W).Spectroscopic redshifts are available for 748 sources in the CEERS catalog, which were collected from literature surveys of DEEP2/3 (Cooper et al. 2011;Newman et al. 2013), Masters et al. (2017), MOSDEF (Shivaei et al. 2018), and SDSS DR15 (Aguado et al. 2019).A comparison of the photo-z estimates to the spectroscopic redshifts yield negligible systematics and a dispersion of ∆z/(1 + z) = 2.68%.
For validation, the candidates have been examined from an independent photometric extraction by Weaver et al. (in prep.) using the Grizli mosaics that are point spread function PSF-matched to F160W.Sources were identified from a noise-equalized stacking of F277W, F356W, and F444W using SEP.Photometry was extracted using SEP in 0.7 diameter circular apertures on all available HST and JWST bands.Following Whitaker et al. (2011), noise properties were measured in >1000 empty apertures.Photometry and associated uncertainties were then corrected to the total flux based on kron radii measured on the F444W mosaic.The two versions of photometry are found to be consistent, and we continue to use the public catalogs from Brammer et al. in following analysis.

Selection
CGG-z5, a compact galaxy group centered at RA 214.812 and Dec 52.8368, was selected as the highest galaxy overdensity at z > 2 in the JWST catalogs produced by Brammer et al. Using GalCluster (Sillassen et al. 2022), we mapped the distance of the fifth nearest neighbor Σ 5th in redshift bins at z = 2-9 in JWST fields of 208 arcmin 2 , including CEERS-EGS, GLASS, SMACS-0723, Sunrise arc, J1235, MACS-0647, Windhorst NEP TDF, NEP 2MASS star, and Stephan's quintet.In order to filter out largely uncertain photometric redshifts, z phot , in each redshift bin, z bin , only sources with a 16th percentile of z phot z 16th > z bin − 8% × (1 + z bin ) and z 84th < z bin + 8% × (1 + z bin ) were preselected for overdensity analysis -with 8% being the 3σ uncertainty of 1 + z from comparison to spec-z (see Sect. 2.1).CGG-z5 is found with the highest overdensity in Σ 5th which is 6.6σ larger than the mean galaxy density at 5 < z < 6 in the CEERS field.As shown in Fig. 1, it hosts a compact group of galaxies, where six candidate members with 5.0 < z phot < 5.4 are assembled in a 1.5 × 3 area (∼10×20 kpc 2 at z = 5.2).We note that the six sources were first identified in F444W and are not blended in the image (Fig. 1).The shorter wavelength photometry was measured within a fixed aperture on the position of F444W sources, thus the photometry measurements are free of blending issues and reliable for this sample.To double-check this, we adopted an alternative overdensity analysis algorithm from Brinch et al. (2022) and reconfirmed the high galaxy overdensity in CGG-z5.
We note that EAZY is a nonparametric code that fits SEDs using a linear combination of 12 preselected flexible stellar population synthesis (FSPS) templates.It is efficient at deriving photo-z for large samples, while further detailed modeling is required for deriving physical parameters of individual sources.To this end, we ran Bagpipes (Carnall et al. 2018) SED fitting with JWST+HST photometry.Following Heintz et al. (2022), we assumed a constant star formation history (SFH) and the attenuation curve of Salim et al. (2018).We also adopted radiation fields in the range of −3 < log U < −1, a metallicity grid from 0 to solar metallicity (Z ), an A V grid from 0 to 2, and an age grid from 1 Myr to 2 Gyr.We found excellent agreement in z phot between Bagpipes and EAZY (Fig. A.1).We thus reran Bagpipes adopting the photometric redshifts as derived by EAZY.The results are listed in Table 1.We note that the stellar masses from Bagpipes are ∼×1.5 lower compared to those inferred with EAZY for which the difference is caused by the different sets of templates and algorithms.We adopted this 0.18 dex difference between the two codes as a systematic uncertainty in our stellar masses.

A compact galaxy group at z ∼ 5.2
We present the JWST NIRCam color images of CGG-z5 in Fig. 1, and show HST and JWST images in Fig. 2. The six candidate members all drop out in HST F435W and F606W data, they start to emerge in F814W, and are all well detected in NIRCam bands with similar colors.These properties are consistent with that of Lyman break galaxies (LBGs) at z ∼ 5.
In Fig. 3, we show the best-fitted SEDs from EAZY.The 14 band photometry of JWST+HST are well fitted with a single peak at z ∼ 5.2 in the redshift probability distribution function PDF(z).All of the six galaxies agree well at z ≈ 5.2 within a 1σ uncertainty (Sect.2.2).We note that the excesses in F277W and F410M exhibited in the six SEDs support the z ∼ 5.2 solutions, as the F277W band brackets multiple strong lines [OII]+Hγ+Hβ+[OIII] and F410M flux is boosted by strong Hα+ [NII].The best-fitted templates are all LBG-like with low dust attenuation, which is consistent with the nondetection in MIPS, Herschel, SCUBA2, and radio images.For a sanity check, we compared our results to two alternative versions of photo-z, L11, page 2 of 8 ).These together suggest that CGG-z5 is a compact group of galaxies at z ∼ 5.2.
We list the results of EAZY and Bagpipes SED fitting results in Table 1.The most massive member CGG-z5.ahas a stellar mass M * ∼ 10 9.8 M and a star formation rate (SFR) ∼50 M yr −1 , placed ∼2× above, yet still within the scatter of the main sequence at z ∼ 5 (Schreiber et al. 2015).The satellites around CGG-z5.a are less massive down to log(M * /M ) ≈ 8.4 with low SFRs ≈2−12 M yr −1 .The stellar mass ratios between the center and satellite galaxies are larger than 4:1, suggesting minor mergers.Interestingly, CGG-z5.a is resolved into three components in the high resolution NIRCam images (Fig. 1, left), and CGG-z5.d is resolved into two, suggesting ongoing mergers, star-forming clumps, or extra low-mass satellites.
CGG-z5 has some unique features with respect to previously identified compact structures.For example, compared to the compact group HPC1001 at z = 3.61 (Sillassen et al. 2022, and in prep.),CGG-z5 has a higher projected number density by a factor of ∼3 and is less massive by an order of magnitude in stellar mass.AzTEC5, a starbursting group at z ∼ 3.6 hosting four candidate members (Gómez-Guijarro et al.  has a total stellar mass and a SFR that are 3× and 20× higher compared to that of CGG-z5, respectively.Another interesting case is the z = 2.94 SDSS J165202.64+172852.3, which was recently observed by the NIRSpec IFU (Wylezalek et al. 2022).This structure hosts a red quasar and several companion galaxies in a scale of 10-15 kpc.However, CGG-z5 is at a higher redshift, hosts hosts two times more galaxies (accounting for the resolved substructures of CGG-z5.a and d, see Fig. 1, left), and the center galaxy is less massive than the quasar by 1.4-2.4orders of magnitude.Recently, a lensed protocluster at z ∼ 7.9 was discov-ered with JWST in the Abell 2744 field (Morishita et al. 2022), where the most overdense region contains six galaxies within a projected distance of 60 kpc.It is at much higher redshift, but still less compact than CGG-z5 by a factor of 20 in the projected comoving number density.

Detectability
One could contemplate why this structure was previously undetected despite the breadth of the existing ancillary data.To In contrast, the state-of-the-art JWST imaging detects and resolves six members at seven wavelengths (1.15-4.44 µm), including three low-mass members previously undetected in HST images.JWST photometry significantly improved the constraints on the rest-frame UV slope and Balmer realm, while the nondetection in HST/F435W and F606W anchored the Lyman break.These combined remarkably reduced the photo-z uncertainties.Moreover, the F277W and F410M bands perfectly bracket the features of strong emission lines at z ∼ 5, and the implicit flux excesses in the two bands further narrowed the redshift range.In Fig . A.2, we compared the PDF(z) with and without JWST data, demonstrating that the identification of such faint structures, just one billion years after the Big Bang, can only be achieved with JWST.

Discussion
Admittedly, the nature of such a compact and relatively lowmass structure at z ∼ 5 is unclear, with a range of possible evolutionary stages and paths: (1) a merging system where the low-mass satellites will merge into the central member of the structure, eventually forming a single, massive galaxy; (2) a forming cluster core, which keeps growing via merger and cold gas accretion (Dekel et al. 2013;Daddi et al. 2021Daddi et al. , 2022) ) and eventually forms a massive protocluster by z = 1; and (3) a filamentary structure of galaxies in the line of sight.While spectroscopic follow-up observations (e.g., Keck/DEIMOS and JWST/NIRSpec) are instrumental to disentangle between these scenarios, we can resort to hydrodynamical simulations in the meantime.
In order to better understand the nature and evolution of CGG-z5, we searched for similar structures in the Eagle simulations (Crain et al. 2015;Schaye et al. 2015), a suite of smooth particle hydrodynamic (SPH) simulations.The free parameters of the model were calibrated to reproduce the stellar mass function, stellar mass -black hole mass relation, and the galaxy disk sizes at z ∼ 0.1.The model was also shown to reproduce many low-high redshift observational results which were not used in the calibration (e.g., Furlong et al. 2015;Trayford et al. 2015;Katsianis et al. 2017).For this work we used the fiducial reference volume, a periodic box of 100 comoving megaparsecs per side.To find similar structures, we first searched for galaxies more massive than M * > 10 9.5 M at z ∼ 5. From the sample, we then selected those with more than three nearby galaxies within a 30 proper kiloparsec radius.Only nearby galaxies with stellar mass greater than 10 7 M (all the galaxies have more than 100 stars and gas particles combined) were considered as companions.With this selection criteria, we found 14 such structures in the Eagle reference volume 2 .We traced their evolution with time by following the most bound star particles belonging to the different structures toward lower redshift, while the progenitors were traced using the most bound dark matter particles.As an example, in Fig. 4 we show the one with closest resemblance to CGG-z5; it contains seven members and the two most massive members have stellar masses similar to those found in CGG-z5.
As shown in Fig. 4, the simulated structure has a halo mass of log(M halo /M ) = 12.1 at z = 5, which is consistent with the estimate of 0.9 × 10 12 M for CGG-z5 from the stellar-mass-tohalo-mass relation in Shuntov et al. (2022).Intriguingly, most of these members merge into the central galaxy by z = 4, and form a single galaxy with log(M * /M ) ∼ 10.7.The stellar mass will keep growing to log(M * /M ) ∼ 10.8 at z = 3 and log(M * /M ) ∼ 11.5 at z = 1.The other 13 simulated structures share the same fate, merging into a single, central, massive galaxy by z = 4 − 3 with log(M * ) 11 at z ∼ 1.It is thus tempting to speculate that the CGG-z5 system will also follow a similar evolutionary path, and we captured the process of satellites merging into a single massive galaxy.
It is also interesting to notice that the merging timescale in the simulations is ∼400 Myrs, which is in line with the fact that similar compact and overdense structures (i.e., with more than five members and overdensity Σ 5th > 5σ) are not identified at lower z nor at higher z in the JWST catalogs.Looking at Fig. 4, this can be explained if similar structures have merged into a single galaxy or group with fewer members by z ∼ 4 and if, at z > 5, JWST observations are not deep enough the detect satellite galaxies of lower M * , and therefore not meet our adopted Σ 5th overdensity criteria.This again supports the idea that CGG-z5 is in an early and short-lived phase of massive galaxy formation before coalescence, a so-called proto-massive galaxy.
We should stress that similar overdense structures, usually associated with submillimeter galaxies (SMGs) or qusars (e.g., AzTEC5, Gómez-Guijarro et al. 2018;SDSS J165202.64+172852.3, Wylezalek et al. 2022), have been found at lower redshifts (z ∼ 3−5) hosting more massive galaxies (log(M * ) ≥ 11) and with higher SFRs with respect to CGG-z5, however.According to the evolutionary picture described above, these structures at z ∼ 5 should host an even higher overdensity of galaxies compared to CGG-z5 and should be detectable with JWST.Given the low number density of these massive structures, it is not surprising that we fail to detect any of those in our volume limited observations.Indeed, based on thse number density of SMGs in the AS2UDS sample 4 deg −2 at z > 4 (Stach et al. 2019;Gullberg et al. 2016;Riechers et al. 2020), the expected number of z > 4 SMGs in the volume covered by the JWST observations in this study is <1.The upcoming COSMOS-Web JWST survey, covering 0.54 deg 2 (Casey et al. 2022), is a expected to capture such structures.
An interesting question that arises is whether the merging groups, such as CGG-z5, are the main avenue for the formation of massive galaxies at z ∼ 2−3.We can attempt to gain some initial insights by comparing the space density of overdense structures at z ∼ 5, with the number density of massive galaxies at z ∼ 3. Adopting a redshift bin of 4.5 < z < 5.5, we find that the space density of galaxies with log(M * /M ) > 9.8 and ≥3 companions within a radius of 30 kpc is 8×10 −2 cMpc −3 .On the other hand, in our JWST catalogs, we have identified twenty galaxies at 2.9 < z < 3.1 with log(M * /M ) > 10.8, corresponding to a space density of 0.12 cMpc −3 .While hampered by uncertainties (e.g., cosmic variance) and selection biases, the rough agreement between the two estimates makes merging groups appealing as the main progenitors of massive galaxies at cosmic noon.L11, page 8 of 8

Fig. 2 .
Fig. 1.JWST RGB color images of CGG-z5 at z ∼ 5.2.Left: color image composed by NIRCam F115W (blue), F150W (green), and F200W (red) data.Right: color image composed by NIRCam F277W (blue), F356W (green), and F410M (red) data.Candidate members are labeled with text in orange.The two color images are matched in the World Coordinate System, and the RGB frames are composed in linear scale with identical limits.

Fig. 3 .
Fig.3.SEDs of candidate members in CGG-z5.JWST+HST photometry is shown via data points with error bars (black: S /N > 2; light gray: S /N < 2), which are best fitted by galaxy templates in blue.The uncertainty of the best-fit template is shown in light blue, and the best template flux in filters is shown with filled circles.On the right side of each panel, we show PDF(z) from EAZY with the best-fitted z phot labeled.

Fig. 4 .
Fig. 4. Stellar mass map of a "CGG-z5"-like group in the Eagle simulations.Stellar mass of individual galaxies are shown in the legend with halo mass M halo = (M 200 /M ).The simulations suggest that CGG-z5 will form a single massive galaxy by z = 3.
Fig. A.1.Redshift comparison.Upper: Comparison of EAZY photo-z computed from two versions of photometry: Weaver et al. and Brammer et al.Bottom: Bagpipes vs. EAZY photo-z, which were all computed using photometry from Brammer et al.Group members are labeled with green text.

Table 1 .
Candidate members in CGG-z5.Notes.The z phot results are from EAZY SED fitting with JWST+HST photometry, and the remaining parameters were computed by Bagpipes at the EAZY z phot assuming a constant SFH and a Salim attenuation curve.Errors are quoted from the 16th and 84th quartiles without including a systematic uncertainty of ∼0.18 dex in stellar mass.