© The Authors 2024

1. Introduction

Galaxies grow over cosmic time through star formation fed by cold gas accretion drawn from a reservoir supplied by the surrounding ‘intergalactic medium’ and neighbouring galaxies. The number and frequency of accretion events shapes the visible properties of present-day galaxies, such as their stellar mass, type (morphology), colour, and size. Key insights into the underlying physics of galaxy growth in this scenario are obtained from correlations (‘scaling laws’) among these fundamental and physical parameters of galaxies over scales ranging from low mass dwarfs to massive giants. Most of these properties are correlated with the host ‘environment’ when defined as a local density using the number of surrounding galaxies over a fixed volume (see Muldrew et al. 2012). For example, the well-established ‘morphology–density’ relation (Dressler 1980; Blanton & Berlind 2007) shows that redder, elliptical galaxies are more likely found in high density clusters. The environmental impact on size growth, however, has remained a mystery.

The most common parameter used to examine this issue is the effective, or half-light, radius. The effective radius is defined at the location that encloses half the total light of a galaxy (r_e; de Vaucouleurs 1948). When plotted against stellar mass (M_⋆), the resulting intrinsic scatter in the r_e − M_⋆ plane is large (∼0.3 − 0.5 dex at low redshift; Shen et al. 2003), consequently showing little to no distinction for galaxies across varied environments and stellar mass (Fig. 1; see also Huertas-Company et al. 2013; Kuchner et al. 2017). Including the r_e of dwarf galaxies with M_⋆ < 10⁹ M_⊙, which are known to be the most affected by the environment (Elmegreen & Hunter 2017) does not significantly change these findings.

Fig. 1. Visualisation of the problem with using effective (or half-light) radii as a size definition. A massive Milky Way-like galaxy (left) and an iconic low surface brightness, low mass dwarf galaxy (DF44; van Dokkum et al. 2016) (right), frequently thought to be as extended as the Milky Way, are shown at the same physical scale using images of similar sensitivity. In spite of the approximately four-orders-of-magnitude mass disparity, both of these galaxies have a comparable half-light radius (re; colored contours), partly explaining the large scatter in the re−mass relation and demonstrating that conventional measures of light distribution can be deceptive characterisations of total galaxy extension (Chamba et al. 2020). Thus, re may not be ideal to trace the environmental impact on size and its dependance on light concentration explains the lack of consensus in the literature.

These results are puzzling, given that many other physical properties of a galaxy are fundamentally affected by the environment. The large scatter is also problematic for the cosmological predictions of the dark matter halo spin, angular momentum, and mass, which set the size of a galaxy. The main issues are that the scatter in the r_e − M_⋆ plane is difficult to reproduce and that the stellar-to-halo mass relation in the dwarf regime from different models diverge. The divergence makes the determination of the minimum halo mass where galaxies can form stars highly model dependant (see Wechsler & Tinker 2018).

Chamba et al. (2020) have shown that the large scatter in the r_e − M_⋆ plane can be explained by the fact that r_e is insensitive to the total extent of galaxies. This is evident in the comparison between massive galaxies and low mass, diffuse low surface brightness (LSB) dwarfs (see Fig. 1 here and their Fig. 5). Consequently, various galaxies can have a similar r_e but fundamentally different masses (see also the reviews by Graham 2019; Chamba 2020). This fact is due to the dependence of r_e on how concentrated the light is, further complicating the interpretation of the scaling relation (D’Onofrio et al. 2021). Decades of research on the use of r_e alone to address the problem of the environment on galaxy size has produced ambiguous results (see Behroozi et al. 2022), and its scatter issues are unresolved.

Fortunately, r_e is not the only radius parameter at our disposal. The development of deep imaging techniques and surveys have allowed for a more accurate characterisation of the low surface brightness outskirts of galaxies (Malin 1978; Bland-Hawthorn et al. 1993; Calcáneo-Roldán et al. 2000; Mihos et al. 2005; Martínez-Delgado et al. 2010; Duc et al. 2011; Abraham & van Dokkum 2014; Capaccioli et al. 2015; Fliri & Trujillo 2016; Higgs et al. 2016; Rich et al. 2019; Bílek et al. 2020; Annibali et al. 2020; Trujillo et al. 2021; Ossa-Fuentes et al. 2023, see Fig. 11 in Euclid Collaboration 2022 for a comparison between current and future image/survey depth from space and ground based efforts since the Sloan Digital Sky Survey York et al. 2000 era). Taking advantage of such data, it is possible to propose a new, physically motivated definition of galaxy size that is more representative of the boundaries of galaxies. Based on theoretical (e.g. Schaye 2004) and observational evidence (in the ultra-violet, see Martínez-Lombilla et al. 2019, 2023), Trujillo et al. (2020) defined the size of a galaxy as ‘the radial location of a star formation threshold’. This definition is physically motivated because it is directly linked to one of the main growth channels of galaxies, namely in situ star formation and the existence of a gas (or stellar) density threshold below which stars are unable to form in a galaxy.

The currently used tracer, or proxy, for the Trujillo et al. (2020) size definition that is suitable for general purpose multi-band optical imaging surveys, including the Sloan Digital Sky Survey (SDSS; York et al. 2000), is the truncation radii. The truncation is the radius corresponding to an outermost change in slope or a cut-off in surface brightness or surface stellar mass density radial profile in a galaxy; it was discovered by van der Kruit (1979). For operative reasons, Trujillo et al. (2020) first used a fixed surface stellar density of 1 M_⊙/pc² to measure galaxy size, motivated by the typical density of truncations in Milky Way-like galaxies (Martínez-Lombilla et al. 2019). The physical properties of the truncation feature, such as its colour and stellar surface density, however, depend on the total stellar mass and morphology of galaxies (Chamba et al. 2022). This finding is a reflection of the varied evolutionary histories of galaxies and led Chamba et al. (2022) to define the ‘edges’ of galaxies more generally as ‘the outermost radial location where a significant drop in (either past or ongoing) in situ star formation is found in a galaxy due to the existence of a star formation threshold’. This edge definition can be used as a physically motivated size measure similar to Trujillo et al. (2020). The impact of the environment on this edge size measure is, however, still unknown, and it is the main scientific interest of this paper. In Sect. 2, we discuss in more detail the properties of the edges found in Chamba et al. (2022).

In contrast to these new size definitions, it is interesting to point out that r_e was historically fixed to enclose half the total light and not another fraction by de Vaucouleurs (1948) mainly to operatively parameterise the light profiles of elliptical galaxies (see also Graham 2019). There is no evidence in the archives that suggests that the definition itself is linked to the physics of how galaxies grow in size. The choice of r_e was probably motivated by the difficulty of measuring galaxy extensions in the past.

In fact, similar arguments can be made about size measures based on fixed surface brightness levels (Redman 1936; Holmberg 1958). Those levels were pragmatically chosen to be near the typical depth of the (shallower) photographic plates used. With this method, as much of the total galaxy light detected as possible was enclosed but only on a per image or observation basis (see the historical review in Chamba 2020).

Although pragmatically justified in the past, these historical choices or measurements are arbitrary when considering that current images are far more homogeneous and deeper, clearly showing us that the past measures may in fact be biased against the galaxy outskirts that can now be detected. Our goal is thus to make use of the wide availability of deep images and measure a galaxy size that is based on one of the main formation channels and independent of a prior (i.e. an arbitrarily selected light fraction or surface brightness level).

Using this physical approach, the Trujillo et al. (2020) and Chamba et al. (2022) measurements are a better representation of the total extensions or boundaries of galaxies in a wide stellar mass range compared to r_e, overcoming the issues visualised in Fig. 1. Remarkably, when the measure is used to define the size–stellar mass relation, the intrinsic scatter is very small: ∼0.06 dex, which is one-third of that resulting from r_e or other size metrics. These improvements compared to previous size measures make the edge a promising approach to address the environmental problem on galaxy growth.

In this paper, we use the deep, multi-band optical imaging from the Fornax Deep Survey (Venhola et al. 2017; Iodice et al. 2019) and DESI Legacy Imaging Surveys (Dey et al. 2019) to measure the sizes of galaxies in three different environments: the Fornax galaxy cluster (a high density environment), confirmed satellite galaxies in nearby < 40 Mpc Milky Way analogues, and galaxies outside clusters or comparatively in lower density environments and located at distances < 100 Mpc (i.e. those in the field).

While we particularly focus on dwarf galaxies because they are the most impacted by the environment (e.g. Elmegreen & Hunter 2017), the more massive galaxies in Fornax and our satellite galaxy samples are also considered for completeness. Instead of using r_e, we compare their edge properties and resulting edge radii–mass relations to address the impact of the environment on size. These findings are compared with those traditionally resulting from the use of the effective radius.

The paper is organised as follows. Sect. 2 summarises the properties of edges and the associated size-mass relation outside clusters (or in field environments), particularly for dwarf galaxies. Sect. 3 describes the catalogues and data used for this work for the different samples considered. Sect. 4 provides the details of the image processing, radial profile creation, and edge identification procedure. The results are described in Sect. 5 and finally discussed in Sect. 6. Throughout this work, we assume a flat Λ-CDM cosmology with H₀ = 70 and Ω = 0.3.

2. Characteristics of edges in the nearby Universe

As mentioned in the Introduction (Sect. 1), Chamba et al. (2022) defined the ‘edge’ of a galaxy as ‘the outermost radial location where a significant drop in (either past or ongoing) in situ star formation is found in a galaxy due to the existence of a star formation threshold’. In this section, we summarise the currently known characteristics of galaxy edges in the nearby Universe (redshift z < 0.09) studied in Chamba et al. (2022), which we later use as a reference sample in our analysis.

Criteria to identify the edge. As a proxy for the location of the edge, Chamba et al. (2022) used the location of the truncation or the outermost change in slope in the radial surface brightness, colour and stellar mass surface density profiles of the field galaxies. This feature has previously been connected to star formation thresholds where in situ star formation is cut-off (e.g. Schaye 2004; Martínez-Lombilla et al. 2019; Díaz-García et al. 2022) and is therefore currently the strongest signature of the edge detectable in optical imaging. We use the same signature in this work.

We visualise the locations of galaxy edges for a number of example field galaxies with stellar masses ∼10⁹ M_⊙ from the Chamba et al. (2022) sample (the median mass of the range considered in that work) in Fig. 2. These galaxies are categorised as ‘dwarfs’ in stellar mass. The location of the detected edges are shown as vertical dashed lines in each profile panel. The surface brightness of the edge typically corresponds to ∼26.5 − 27 mag/arcsec² in the SDSS g-band (see their Appendix B, which shows a detailed analysis on the effect of image depth on edge detection).

Fig. 2. Radial profiles of galaxies with stellar masses ∼109 M⊙ from the Chamba et al. (2022) field sample. Left to right: RGB composite image from IAC Stripe 82 with a grey background to highlight the low surface brightness edge location (pink ellipse). Surface brightness profiles in r-band (orange) and g-band (cyan), g − r colour and surface stellar mass density profiles. The location of the detected edges are shown as vertical dashed lines and the shaded grey areas are the typical uncertainties associated with repeated measurements by Chamba et al. (2022) (∼0.04 dex).

The shaded grey areas in the panels are the typical uncertainties associated with repeated identifications (∼0.04 dex). The repeated identifications were performed by the authors using the same data and profiles. The quoted value is the typical deviation between the repeatedly located edge radii. The technical details on how radial profiles are derived and corrected for inclination is described under the Methodology (Sect. 4).

Notice how the colour profile beyond the located edges can either turn redder or bluer. As discussed in Chamba et al. (2022), for dwarf galaxies the different colour profile shapes are indicative of either inside-out or outside-in star formation (see also the work by Herrmann et al. 2016). The Chamba et al. (2022) criteria for more massive galaxies, however, is slightly different, given that these galaxies have significantly varied evolutionary pathways.

Apart from the change in slope, late-type or spiral galaxies often have a ‘U’ shape colour profile (Bakos et al. 2008; Azzollini et al. 2008). Chamba et al. (2022) has shown that the reddening of the colour profile signifies both a drop in in-situ star formation when traced using the truncation in ultraviolet observations as well as the stellar halo component or migrated stars from the inner disc (see their Fig. 2).

In the case of early-type or elliptical galaxies, the most clearest criteria for the edge reported in Chamba et al. (2022) is in the colour profile where there is a very rapid drop towards bluer colours (slopes compatible with ∼ − 5 in some extreme cases). These bluer colours potentially represent a more recently accreted stellar component that makes up the outer envelope in these galaxies. Depending on the morphology in the total sample considered here, the above features were collectively used when identifying the edges.

Edge size-mass properties. The edges of the 180 dwarfs from the Chamba et al. (2022) sample have an average stellar surface density of 0.6 M_⊙/pc² and their sizes follow the relation $R_{\mathrm{edge}}\sim M_{\star }^{1/3}$ with an observed scatter of ∼0.09 dex. The average density of edges in the full sample including the more massive galaxies is ∼1 M_⊙/pc², similar to the value found for nearby Milky Way-like galaxies (Martínez-Lombilla et al. 2019). The elliptical galaxies have even higher edge densities > 3 M_⊙/pc². The lower edge density in dwarfs compared to the massive galaxies is consistent with the fact that dwarfs are less efficient in forming stars (Huang et al. 2012) and massive ellipticals are known to be highly efficient in their star formation at earlier times (e.g. Jaskot et al. 2015).

Additionally, late-type bluer, younger galaxies were found to be larger in size at a fixed stellar mass compared to redder, older ones. These bluer galaxies are in the range between Sb-Scd morphological types while the redder ones are S0/Sa types (see Trujillo et al. 2020). This result was not definitive for galaxies with M_⋆ ≤ 10⁹ M_⊙, given that the morphological types of these lower mass galaxies studied in Chamba et al. (2022) are not available. Nevertheless, this colour (age) bifurcation in the size-mass relation potentially indicates that the older, redder galaxies reached their present-day size at an earlier epoch compared to younger, bluer ones. In this work, we will analyse the environmental influence on the edge locations, densities, galaxy colours and morphology for comparison with the above results from Chamba et al. (2022).

3. Data and sample selection

We selected a volume limited sample of dwarf galaxies in order to probe a wide range in stellar mass and environmental conditions. We classify galaxies as ‘cluster’ based on their membership in the Fornax Cluster, as ‘satellite’ if the galaxies are in a low density group environment but confirmed to be associated with a nearby host, and as ‘nearly isolated’ for all other galaxies in groups or in isolated environments. Throughout this work, we use ‘field’ as a reference to galaxies outside a cluster environment. In our study, the field includes our satellite and nearly isolated dwarf samples. We describe the catalogues and data used in Table 1 in the following subsections.

3.1. Cluster environment

Fornax is the second most massive cluster within 20 Mpc and has a virialised mass of 7 × 10¹³ M_⊙ (Drinkwater et al. 2001a). The cluster has been well-studied using multi-band surveys from the ultra-violet to radio wavelengths (e.g. Martin & GALEX Team 2005; Davies et al. 2013; Eigenthaler et al. 2018; Venhola et al. 2018; Zabel et al. 2019; Loni et al. 2021; Serra et al. 2023). The most up-to-date catalogues of the cluster include 795 low mass galaxies with M_⋆ ≲ 10⁹ M_⊙. These galaxies include 265 low surface brightness dwarfs (Venhola et al. 2018, 2022; Su et al. 2021). An additional 42 more luminous early- and late-type galaxies are found in the main cluster and Fornax A sub-group (ETGs and LTGS, respectively; Iodice et al. 2019; Raj et al. 2019, 2020), spanning a total stellar mass range between 10⁵ − 10¹¹ M_⊙. The locations of all the galaxies from the Su et al. (2021) catalogue are shown in Fig. 3.

Fig. 3. Schematic of the Fornax Cluster and Fornax A sub-group from the Fornax Deep Survey. The positions of dwarfs (purple stars), late-type and early-type galaxies (blue crosses and red dots, respectively) catalogued by Venhola et al. (2018) are plotted. The virial radius of the cluster (black circle with radius ∼2 deg) and Fornax A sub-group (black, dashed circle with radius ∼1 deg). The late-type galaxies in the subgroup are plotted as blue pluses for distinction.

We made use of the deep g and r-band optical images from the Very Large Telescope (VLT) Survey Telescope as part of the Fornax Deep Survey (FDS; Venhola et al. 2017; Iodice et al. 2019; Peletier et al. 2020). Given our goal to search for edges in a large sample of galaxies and features that are typically of low surface brightness ∼25 − 27 mag/arcsec² in the SDSS r-band (see also Fig. B2 in Chamba et al. 2022), the combined depth and pixel scale of the FDS imaging (∼30 mag/arcsec² over boxes of 10 × 10 arcsec² scales) provides an ideal dataset to characterise edges in a high density environment. We use the 582 Fornax cluster galaxies catalogued in Su et al. (2021) in this work. As in previous work (e.g. Raj et al. 2019, 2020), we assume these galaxies are located at 20 Mpc. We do not make any distinction between the ‘cluster’ and ‘Fornax A’ sub-group in this work unless specifically mentioned within the text.

3.2. Group and satellite environment

For comparison with the Fornax galaxies, we also study the Stage II SAGA satellite sample of 127 galaxies around 36 Milky Way analogues out to 40 Mpc (Geha et al. 2017; Mao et al. 2021). The sample is spectroscopically confirmed and predominantly star forming with stellar masses ranging from 10⁶ − 10¹⁰ M_⊙, the majority of which have a stellar mass of ∼10⁸ M_⊙. Public deep photometric imaging catalogues from the Dark Energy Camera Legacy Survey (DECaLs; Dey et al. 2019) are also available for the sample. Similar to FDS, we used the g- and r-band images of DECaLs, which have a depth of ∼28 mag/arcsec² in the g-band and is sufficient for the detection of edges.

To increase our sample in the lower mass regime < 10⁷ M_⊙, we combine the SAGA sample with the ELVES satellites belonging to Milky Way mass hosts within the local volume (< 10 Mpc; Carlsten et al. 2021, 2022). The stellar mass range of ELVES galaxies are between 10^5.5 − 10^8.5 M_⊙. In our work, we only use a sub-sample of the ELVES satellites with the criteria that their distances are confirmed and public DECaLs imaging are available at their locations. These two criteria leads to a sample of 179 ELVES satellite galaxies. Our total satellite galaxy sample from SAGA and ELVES combined consists of 306 galaxies.

3.3. A nearly isolated environment

For a nearly isolated sample, we use the edge measurements made in Chamba et al. (2022) for a wide selection of galaxies within a stellar mass range of 10⁷ − 10¹² M_⊙ using the deep IAC Stripe 82 survey (Fliri & Trujillo 2016; Román & Trujillo 2018). This sample consists of 624 galaxies taken from a combination of the Nair & Abraham (2010) and Maraston et al. (2013) catalogues. 180 of these galaxies are categorised in that work as dwarfs according to their stellar mass < 10¹⁰ M_⊙. The morphological classifications of these ‘dwarfs’ are not available. However, the classifications of the late-type (spiral) and early-type (elliptical) galaxies are available from the Nair & Abraham (2010) catalogue. Only galaxies within the foot print of the SDSS Stripe 82 were studied.

To gain insight on the surrounding environment of these galaxies, we cross match the Chamba et al. (2022) sample with the Yang et al. (2007) SDSS group and cluster catalogues¹. Unfortunately, we are able to find definitive group associations only for 79–83 dwarfs out of the 180 studied in Chamba et al. (2022) as their central coordinates are at least within 1–2 arcsec of those in the Yang et al. (2007) galaxy catalogue. More recent catalogues on the SDSS galaxies do not include the Stripe 82 footprint (see e.g. Lim et al. 2017). However, as this sub-sample covers the full stellar mass range of the Chamba et al. (2022) dwarfs, it is still useful to analyse them. We find that ∼75% of the sub-sample in Yang et al. (2007) are reported as the most massive and single member of their respective group. In other words, they are isolated.

For the rest of the sample, we find that the median angular separation between the dwarf and a nearest galaxy or group from Yang et al. (2007) is 223 arcsec. Upon further investigation, these galaxies or groups are located at significantly different redshifts, with our sample being nearer at a median z ∼ 0.016 ± 0.002 and the groups at ∼0.081 ± 0.04. 84% of these groups are composed of single member galaxies. According to Yang et al. (2007), such galaxies are defined as isolated. Therefore, these results allow us to conclude that the Chamba et al. (2022) dwarfs are either isolated or in low mass groups not characterised in Yang et al. (2007). In other words, the Chamba et al. (2022) dwarfs are nearly isolated or in the field.

Additionally, we used a lower mass Local Volume (LV) sample of dwarfs studied in Carlsten et al. (2021) that overlaps in stellar mass and distance with ELVES. These galaxies were selected as a sub-sample from the Updated Nearby Galaxy Catalogue (Karachentsev et al. 2013); however, we only used galaxies that have DECaLs imaging. This criteria results in a sample of 96 galaxies in addition to the 180 low mass galaxies in the Chamba et al. (2022) sub-sample.

Karachentsev et al. (2013) also estimates tidal indices for these galaxies to describe and quantify their local environment. The index is computed by ranking the nearest neighbours ‘n’ according to the strength of their tidal force on the galaxy, estimated simply by $F_{\text{tidal}}~M_n/D_n^3$ where M_n is the mass of the neighbour and D_n is the distance between the galaxy and the neighbour. Negative values of this index correspond to isolated galaxies. Of the 96 field galaxies we are able to consider here, 78% have negative tidal indices². Therefore, when combining the Chamba et al. (2022) and Karachentsev et al. (2013) sub-samples, the majority of these galaxies can be considered nearly isolated.

Total sample. The total sample of galaxies (Fornax, SAGA, ELVES and field) considered in this work amounts to 1608. Following Trujillo et al. (2020), after removing galaxies that have an inclination or axis ratio < 0.3 to avoid projection effects in the computation of the stellar mass (Sect. 4) and those significantly impacted by neighbouring galaxies (overlapping or due to bright stars), the final sample analysed amounts to 894 galaxies, 17 of which are Fornax massive early- or late-type galaxies.

Table 1 summarises the data and final sample used. Fig. 4 shows the distribution of the number of galaxies in our cluster, field and satellite samples in stellar mass bins of 0.5 dex. While our total isolated and satellite samples consist of about half the number of Fornax cluster dwarfs in the lowest stellar mass bins M_⋆ < 10^7.5 M_⊙ and nearly double at the higher mass bins, we have sufficient galaxies to derive and compare the distributions of these galaxies in the size scaling relations.

Fig. 4. Distribution of the number of galaxies in stellar mass bins of width 0.5 dex in our final Fornax cluster (purple), satellite (yellow, right hatched), and nearly isolated (blue, dotted and left hatched) samples. The field sample combining the satellites and isolated galaxies are plotted in dashed black lines. C22* in this figure indicates that we have included the Chamba et al. (2022) LTG and ETG sample here only to show the lack of cluster galaxies beyond M⋆ > 109.5 M⊙ (grey region).

3.4. Characterisation by morphology and distance

Morphological classification of the dwarf galaxies in this study is only available for the LV, ELVES and FDS sub-samples. These samples are classified as either ‘early’- or ‘late’-type galaxies based on visual inspection. In the case of LV and ELVES, Carlsten et al. (2022) categorise ‘dwarfs with smooth, regular, and generally low surface brightness morphology’ as early-type satellites. Late-type galaxies are those with ‘clear star forming regions, blue clumps or dust-lanes’.

The FDS galaxies are classified similarly. However, Venhola et al. (2018) additionally divide the early-type sample as either ‘smooth’ red galaxies without ‘clearly distinguishable bars or spiral-arms’ or those with structure. The former category includes ‘giant early-type galaxies with no clear structure, nucleated and non-nucleated dwarf ellipticals’. The latter category also consists of red galaxies but they have sub-structure such as a ‘bulge, bar or spiral arms…includ[ing] the S0s and dEs with prominent disk features’. Venhola et al. (2017)’s late-type category includes ‘spirals, blue compact dwarfs and dwarf irregular galaxies’. In our analysis, we treat galaxies with or without sub-structure in FDS as belonging to the same category, broadly defined as early- or late-type.

Morphological classification is not available for the SAGA and Chamba et al. (2022) samples. However, in the case of SAGA, Mao et al. (2021) used a condition on the equivalent width of the H_α line such that EW(H_α) < 2 Å to tag galaxies as ‘quenched’. The galaxy is considered ‘star forming’ in other cases. Using the EW(H_α) to make this distinction is motivated as the H_α line reflects the presence of star formation within the last 5 Myr (e.g. Flores Velázquez et al. 2021).

In the visual comparison between ELVES and SAGA by Carlsten et al. (2022), the above criteria for quenched satellites is shown to be compatible with that of the red ultra-violet (UV)-optical colours of the ELVES early-type classification scheme (see their Fig. 10). However, they identify three exceptional cases in their visual comparison. Relevant to our final sample are two of these galaxies that are visibly blue and irregular but have no detected Hα emission: SAGA galaxy 1237661852013166857 (RAdeg = 206.99, DEdeg = 43.5872) and DES-203016260 (RAdeg = 322.9709, DEdeg = –43.6539). Their stellar masses are log₁₀M_⋆ = 7.9 and 7.7, respectively.

Apart from these two galaxies, SAGA’s satellites listed as star forming also have a similar colour to the ELVES late-type galaxies. For this reason, in our analysis, we treat the SAGA quenched sample as ‘early-type’ and star forming sample as ‘late-type’. Classifying the two exceptional galaxies as late-type do not change our results significantly.

For completeness, we make a similar classification criteria for the Chamba et al. (2022) sample. We obtain the EW(H_α) from the computations made by the Portsmouth Group for SDSS/BOSS galaxies³ produced by Thomas et al. (2013). We impose the same condition as in SAGA to separate quenched and star forming galaxies in this sample. Similar to the SAGA analysis by Carlsten et al. (2022), we treat the quenched sample as early-type and star forming as late-type.

The number of galaxies in our total sample considered as either early or late-type for each environment are listed in Table 2. A few remarks are in order. First, five of the FDS early-type galaxies are from the sample studied in Iodice et al. (2019) and are located in regions of the cluster not dominated by intra-cluster light (ICL). These early-type galaxies have stellar masses 10⁹ M_⊙ < M_⋆ < 10¹⁰ M_⊙. As we required a sky background estimation in our radial profile derivation (Sect. 4), including galaxies within the ICL regions would specifically require a more careful treatment of this component or ad hoc methods such as that shown in Iodice et al. (2019). Therefore, for this paper we only focus on galaxies without ICL in their background.

Second, in the case of the FDS late-type galaxies previously studied in Raj et al. (2019, 2020), we able to study 12 galaxies from their sample. These galaxies have stellar masses > 10⁸ M_⊙. Apart from the ICL issue, we remind the reader that this selection also includes our additional criteria on the inclination of the galaxy and those impacted by neighbouring galaxies (see the paragraph on ‘Total Sample’ in the previous section). Throughout the paper we refer to our sub-sample from Iodice et al. (2019) and Raj et al. (2019, 2020) collectively as ‘FDS ETGs+LTGs’ and FDS ‘dwarfs’ for the rest of the sample from Su et al. (2021). Our FDS ETGs+LTGs sample are predominantly LTG.

Finally, as we use the same nearly isolated sample as in ELVES (Carlsten et al. 2021), we lack early-type galaxies in this category by construction. For this reason, we focus on comparing the sizes of galaxies in different environments primarily at fixed stellar mass. The segregation by morphology for each sample is also shown and discussed where appropriate.

In Fig. 5, we plot the stellar mass distribution of each sample as a function of their distances. We mark the early-type galaxies in orange and late-type in blue. The FDS sample is considered to be at a distance of 20 Mpc. Limiting our analysis to galaxies out to twice the distance of Fornax would almost completely remove the C22 sub-sample. Making such a distance cut would significantly restrict our comparison of edge properties in Fornax to galaxies with M_⋆ ≲ 10^8.5 M_⊙. Above those stellar masses within D ≲ 40, we would have the confirmed SAGA satellites and only ∼25 nearly isolated galaxies from C22.

Fig. 5. Distribution of the sample listed in Table 1 in stellar mass as a function of distance. The points are coloured according to their morphology, broadly categorised here as early- (red) or late-type (blue) [see text for details]. The markers in the legend label each sub-sample. The number of galaxies in each type are listed in Table 2.

Low surface brightness galaxies such as those catalogued in Fornax (Venhola et al. 2022) are not considered in our sample due to the difficulty in measuring the distances of similar galaxies in the field or isolated environments (see e.g. Román et al. 2019; Greene et al. 2022). Future works specifically targeting low surface brightness galaxies within our lower stellar mass bins will readily complement the relations we report here.

4. Methods

We follow the methodology described in detail by Chamba et al. (2022) to derive the surface brightness, colour and surface stellar density radial profiles of the 894 galaxies in our sample. We show the profiles of a number of galaxies with stellar masses ∼10⁹ M_⊙ from the Chamba et al. (2022) sample in Fig. 2. The same methodology is applied to the FDS, SAGA, ELVES and LV galaxies. We summarise the procedures for profile derivation and edge identification for each galaxy below.

Image stamps. We created an image centered on the galaxy using the g and r-band data. The dimensions of the image depends on the diameter of the galaxy. We used three times the Homlberg radius (Holmberg 1958) as a benchmark to create an image that includes the immediate ‘local’ surroundings of the galaxy.

Masking. We summed the r and g-band images and run Sourcerer (formerly Max-Tree Objects; Teeninga et al. 2016; Haigh et al. 2021), a robust segmentation tool, for masking extremely diffuse light sources in an image. We create a ‘master mask’ using the summed image to include source detections from both bands in the segmentation map. The associated sourcerer catalogue includes the best-fit elliptical parameters (axis-ratio and position angle) using the map for each source. We mask every source in the images except the galaxy of interest and the background when deriving radial profiles. Additional manual masking is used when necessary.

Background subtraction. We define ‘background’ as every pixel in the image undetected by Sourcerer. While the survey images are already background subtracted, the subtracted values are not always representative of the more local background variations in the immediate surroundings of galaxies and are either over- or under-subtracted (see Trujillo et al. 2020). Therefore, to make a more precise and representative background subtraction to the galaxy of interest, in our default pipeline we make an additional ‘local’ background estimate. We selected an elliptical annulus between two and three times outside the segmented region of the galaxy. We visually verified that all light sources within this annulus are masked and as mentioned in the masking step, additional pixels are manually masked if necessary. An average local sky background value is then measured using all the unmasked pixels within this annulus. This value is then subtracted from the images. This method of subtraction is similar to the approach developed in Pohlen & Trujillo (2006) as it removes the flat, average asymptotic flux value in the image at least twice the distance beyond the edge of the galaxy.

Radial profiles. We used the elliptical parameters from the segmentation tool to average flux within bins of concentric elliptical annuli beginning from the centre of the galaxy until the end of the background subtracted image. The number of bins we use vary with galaxy extension. We first defined a parameter ‘b_prof’ such that b_prof = 2r_eff, where r_eff is the effective semi-major axis directly taken from the catalogues we use (see Sect. 3) in pixel units. The bin size was then computed by dividing the distance from the centre of the galaxy to 3b_prof (which is well beyond the edge radii of our sample) by the number of bins N_bin. We set N_bin = b_prof rounded to the nearest integer. This gives us a binsize of 3 pixels in our profiles, corresponding to ∼0.79 arcsec in DeCALs and ∼0.63 arcsec in FDS.

Such a binsize is justified if the edge feature occurs over sub-kpc scales. At the distance of the Fornax Cluster, the bin size is 60.5 pc. At the median distance of our field sample (populated by SAGA) ∼34 Mpc, the bin size ∼134 pc. Given that our FDS data is our deepest dataset with the smaller pixel scale, we also test our profile derivation method using different bin sizes in Appendix A for a bright dwarf galaxy in our Fornax sample.

We use fixed elliptical parameters to ensure that the edge we locate is not artificially caused by the use of a different position angle or axis ratio within the galaxy. This step produces the g and r flux radial profiles of the galaxy.

The profiles are then converted to surface brightness (μ_g and μ_r) following the appropriate calibrations of the datasets considered. The g − r and surface stellar mass density Σ_⋆ profiles are also computed. The stellar masses are computed using colour-mass-to-light ratio relations from Roediger & Courteau (2015) as in Trujillo et al. (2020).

Inclination and Galactic extinction correction. Chamba et al. (2022) only considered low inclination galaxies with axis-ratios ≥0.3 to reduce the impact of the point spread function (PSF) and projection effects on the stellar mass estimation. This axis-ratio limit was carefully chosen by considering the inclination correction necessary for estimating the stellar mass using the disk models developed in Trujillo et al. (2020) for galaxies with varied thickness (see their Fig. 2). For galaxies with higher axis ratios, the inclination correction was found to be independent of thickness. We made the same correction on the radial profiles derived in this work for consistent comparison with the Chamba et al. (2022) measurements. Table 1 in Trujillo et al. (2020) list the fitted parameters we use to make this correction. We additionally corrected the radial profiles for Galactic extinction using the A_g and A_r values computed using the NED calculator⁴.

Identifying the edge. As summarised in Sect. 2, the signature of the edge is identified as a change in slope in the outer radial profiles of galaxies. In particular, for field dwarf galaxies, Chamba et al. (2022) reported that the most visible signature is that the colour beyond the edge in the profile may either increase or decrease towards redder or bluer colours, respectively. This shape of the colour profile can be understood in terms of inside-out or outside-in galaxy evolution, depending on whether older or younger stellar populations are found in the outskirts (Herrmann et al. 2016; Crnojević 2017).

The pipeline to search for and identify the edge signature is drawn as a flowchart in Fig. 1 of Chamba et al. (2022). Here we summarise the main steps as follows. We first visually search for similar signatures where the colour drops or rises towards bluer or redder colours in the derived g − r colour profile of the galaxies. We then visually check whether the galaxy is elliptically symmetric at the chosen radial location (R_edge) by overplotting the elliptical contour at the edge on the RGB image of the galaxy to ensure that the truncation is not created by an artefact in the data or a neighbouring source that was unmasked.

If unmasked sources are found, a second masking is performed at the potential contaminant and the radial profiles are re-derived. We search again for the edge in the same manner described above on the new profile. We report edges only in cases where all potential contaminants that we are able to visually verify are masked.

In many cases studied by Chamba et al. (2022) the edge is also visible in the surface brightness and density profiles. While multiple signatures of a change in slope in all three profiles is further confirmation of the edge feature reported here, we point out that in NC’s experience the most obvious signature in the majority of the dwarf galaxies studied in this and previous work is in the g − r colour profile. There are two potential reasons for this outcome.

The first is that in surveys like SDSS where the shape of the PSF in the g and r band are similar, Chamba et al. (2022) has argued that the PSF effect of broadening a galaxy’s g − r radial profile is roughly cancelled out to first order, at least within the locations where edges are found. From this technical perspective, the inflection point in the colour profile is the more robust signature of the edge compared to the ones in the surface brightness profile. While we do not explore deconvolution techniques on our profiles, dedicated effort is underway within the LIGHTS collaboration (Trujillo et al. 2021) to exactly account for these effects on ultra-deep profiles (Golini, in prep.).

The second reason is that a surface brightness profile in a single band cannot be used as an indicator of the varying stellar populations dominating the structure of galaxies at different radii. Since we are interested in locating a physical feature in the galaxy outskirts that can mark the ‘edge’ of the bulk of the stellar material formed in situ, the colour of the galaxy is an important observable to make this task possible.

In fact, in the recent article by Fernández-Iglesias et al. (2024) who use machine learning to automate the identification of galaxy edges using previously analysed Hubble images (see Buitrago & Trujillo 2024), the authors use different combinations of colour images to define the edge and show that the algorithm is able to recover the identifications made by expert observers. As the use of machine learning is beyond the scope of this paper, we refer the interested reader to that work for details on their training dataset and algorithm. For the above reasons, we are convinced that our criteria for identifying edges using the colour profile in addition to the surface brightness and stellar surface density profiles is more reliable compared to only using surface brightness or stellar mass density profiles (see also Herrmann et al. 2016; Sánchez-Alarcón et al. 2023). In Appendix B, we show how our profile derivation and edge identification method performs for four cases in the Raj et al. (2019) sample that are clearly asymmetric.

Effective radii. In addition to R_edge, we also measured the effective radii (r_e) of each galaxy. Instead of computing the average flux in the elliptical annuli as before, we compute the sum in each annulus and create a light growth curve. The curve is cut-off at the surface brightness limit of data used (see Table 1) to measure the total light (or apparent magnitude) belonging to the galaxy in each band. r_e is then defined as the radius enclosing half the light of the galaxy. In Appendix C, we show that the scaling relations and results using the modelled radii values from Su et al. (2021) for Fornax galaxies do not change our main conclusions.

Colour and age. We used the global g − r colour of the galaxy to estimate its age from the E-MILES models (Vazdekis et al. 2012). The model predictions for the SDSS filters using Kroupa Universal isochrones⁵ are provided for seven specific metallicity values ranging from −2.32 ≤ [M/H] ≤ 0.22. As all our measurements account for the outskirts of galaxies, we select an average metallicity value that is representative of the stellar populations in this regime. For simplicity, we adopt a fixed metallicity of [M/H] = –0.71 across the full stellar mass range. This low metallicity value accounts for both, the metal-poor stellar populations found in the faint outskirts of nearby galaxies (Crnojević 2017; Neumann et al. 2021) as well as in observations and simulations of dwarf galaxies (see e.g. Cardona-Barrero et al. 2023).

In Appendix D, we additionally explore our results using alternative metallicity values from E-MILES that follow the Kirby et al. (2013) stellar–mass metallicity relation for dwarf galaxies. While the estimated age values for individual galaxies are different as expected, the colour and age bifurcation we report in this work remains unchanged.

Stellar mass and surface density. The surface stellar density at the edge (Σ_⋆(R_edge)) was obtained from the surface density radial profiles via linear interpolation. The total stellar masses M_⋆ were computed by integrating the surface stellar density profile out to the surface brightness limit of data used (Table 1).

All the average or mean values in edge locations and densities reported were computed by performing 3σ clipping on the distribution of interest. Using 5σ-clipped values does not significantly change our results.

Uncertainty estimation. We computed the uncertainty in the R_edge and Σ_⋆(R_edge) measurements due to background subtraction and stellar mass estimation following the approach described in Trujillo et al. (2020), Chamba (2020) and summarised in Chamba et al. (2022). We perform this task in order to quantify the movement of R_edge in the size–stellar mass plane due to both uncertainties and the edge surface density–stellar mass plane in the case of Σ_⋆(R_edge).

Each galaxy is parameterised by a pair of values (R_edge, Σ_⋆, edge). Fixing R_edge, for each band the surface brightness radial profile is moved up or down by a random value drawn from a Gaussian distribution defined using the previously measured dispersion value of the image background.

The steps to compute the colour and stellar mass surface density profiles were then followed using the ‘moved’ surface brightness profiles, and we inferred the stellar surface mass density at the edge location in the moved profile, that is, ${\mathrm{\Sigma }}_{\star }^{\mathrm{move}}(R_{\mathrm{edge}})={\mathrm{\Sigma }}_{\star ,\mathrm{edge}}^{\mathrm{move}}$. Similarly, we fixed the density at the edge and tracked the associated edge location changes in the moved profiles. In other words, $R_{\mathrm{edge}}^{\mathrm{move}}$ was taken as the value where ${\mathrm{\Sigma }}_{\star }^{\mathrm{move}}(R_{\mathrm{edge}}^{\mathrm{move}})={\mathrm{\Sigma }}_{\star ,\mathrm{edge}}$.

We repeated this process 1000 times. At each step, we recorded ($R_{\mathrm{edge}}^{\mathrm{move}},{\mathrm{\Sigma }}_{\star ,\mathrm{edge}}^{\mathrm{move}}$) and measured the differences (δ_R, δ_Σ) between each ‘moved’ value and our original (R_edge, Σ_⋆, edge) pair. Finally, we computed the dispersion of the distributions σ_δR and σ_δΣ and used these dispersion values as the uncertainty in our measurements due to background subtraction.

In the case of the uncertainty due to stellar mass, we follow the same procedure outlined above except we obtain random values to move the profiles using a Gaussian distribution with a fixed dispersion value of 0.25 dex. This value was chosen because it is the typical uncertainty in stellar masses when computed using mass-to-light vs. colour relations (Roediger & Courteau 2015).

Scaling relations. We used a Huber regression that is robust towards outliers to estimate the best-fit lines and dispersion values in the scaling relations as in Chamba et al. (2022). To estimate how each of the background and stellar mass uncertainties for each galaxy contributes to the ‘global’ dispersion of the size–stellar mass scaling relation σ_back and σ_mass, we used the fact that the best-fit relation to our observations has a non-negligible dispersion (σ_obs; Table 3).

For simplicity, we assumed that the observed dispersion can be expressed as a quadratic sum:

$$\begin{array}{c}\hfill {\sigma }_{\mathrm{obs}}^2={\sigma }_{\mathrm{int}}^2+{\sigma }_{\mathrm{back}}^2+{\sigma }_{\mathrm{mass}}^2+{\sigma }_{\mathrm{vis}}^2,\end{array}$$(1)

where σ_int is the intrinsic dispersion of the relation and σ_vis is the typical uncertainty from visual identifications (0.04 dex; Chamba et al. 2022). We move each observed profile such that the data point (R_edge, M_⋆) is exactly on or crosses the best-fit relation and infer (R_cross, M_⋆, cross). This process sets ${\sigma }_{\mathrm{back}}^2={\sigma }_{\mathrm{mass}}^2={\sigma }_{\mathrm{vis}}^2=0$ and leaves an artificial ‘intrinsic’ dispersion of the (R_cross, M_⋆, cross) plane, which is the total observed dispersion, as that is the typical displacement of our observed data points to cross the best-fit line. Let us call this artificial ‘intrinsic’ dispersion as σ_Δ. We then randomly draw a new $R_{\mathrm{edge}}^{\mathrm{rand}}$ value from a Gaussian distribution centred on R_edge with the associated σ_δR from the procedure described above and record its movement on the (R_cross, M_⋆, cross) plane. In this case, we assumed that that final dispersion in this plane is

$$\begin{array}{c}\hfill {\sigma }_{\delta \mathrm{cross}}^2={\sigma }_{\mathrm{\Delta }}^2+{\sigma }_X^2,\end{array}$$(2)

where X is the uncertainty we seek to estimate, considering which σ_δR value is used. The estimated uncertainty on the scaling relation due to the background and mass is then separately computed as the difference in quadrature $\sqrt{{\sigma }_{\mathrm{\Delta }}^2-{\sigma }_{\delta \mathrm{cross}}^2}$. For simplicity, we fixed σ_Δ to the total dispersion of the observed relation 0.13 dex (Table 3) as it is the typical displacement of the profiles in the size–mass plane to cross the best-fit line.

For comparison, we follow the same procedures as described above to estimate the dispersion in the effective radius–stellar mass relation contributed by the uncertainties in image background subtraction and stellar mass estimation in our measurements. However, here we mark each galaxy using the (r_e, Σ_⋆, eff) pair of values where Σ_⋆, eff = Σ_⋆(r_e) is the surface stellar mass density at r_e. While we do not analyse the Σ_⋆(r_e)−M_⋆ plane in this work as we focus on the edge properties of galaxies, we use this parameter here for practical purposes in order to obtain $r_e^{\text{move}}$ such that ${\mathrm{\Sigma }}_{\star }^{\mathrm{move}}(r_e^{\mathrm{move}})={\mathrm{\Sigma }}_{\star ,\mathrm{eff}}$. We refer the reader to Stone et al. (2021) for alternative methods of estimating the intrinsic scatter of scaling relations.

5. Results

5.1. Radial profiles

We display the identified edges over plotted on the gri-colour composite images and radial profiles derived in this work for a number of galaxies in our sample in Figs. 6 and 7. To highlight the inflection point we mark as the edge feature, we additionally plot a least squares polynomial fit of degree four in the colour and stellar surface density profile panels as a solid line. Using a polynomial of degree three does not significantly change these fits. We only plot them in these panels for visualisation purposes. We do not use these fits for any of our measurements. While we show these data only for a handful of galaxies as in Fig. 2, we make the profiles and measurements made in this work publicly available as catalogues associated with this publication.

Fig. 6. Similar to Fig. 2: example radial profiles of cluster dwarf galaxies with M⋆ ≲ 108 M⊙. The first panel is the gri-colour composite image using FDS images (Venhola et al. 2018), overlaid on a grey scale background to highlight the low surface brightness boundaries of these galaxies. White in this scale indicates non-detections or background pixels in the data. FDS names for each galaxy from Venhola et al. (2018), Su et al. (2021), the estimated stellar mass in units of log M⋆/M⊙ (left) and edge locations (dashed ellipses) are indicated in each panel. The second, third and fourth panels are the g (cyan) and r-band (orange) surface brightness, g − r and Σ⋆ profiles respectively. The identified edge feature is indicated as a vertical dashed line. To highlight the inflection point we mark as the edge feature in the colour and stellar surface density profiles, we additional plot a least squares polynomial fit of degree four as a solid line in those panels. Galaxies are ordered by increasing stellar mass from the upper to lower panels.

Fig. 7. Similar to Fig. 6 but for satellite galaxies from ELVES Carlsten et al. (2021) within DECaLs (Dey et al. 2019).

5.2. Scaling relations

The resulting size–stellar mass relation using the edge locations are shown in the left panel of Fig. 8. The left panel of the same figure shows the associated edge density-stellar mass relation. The best-fit relations in each panel are also shown along with their intrinsic dispersion (∼0.06 dex for the edge radii) in a lighter shade. The samples considered in this work are labelled in the legend: Fornax (Su et al. 2021, dwarfs in purple pentagons and early- and late-type galaxies in dark blue circles), field (Karachentsev et al. 2013; Chamba et al. 2022, in blue diamonds) and satellites (Mao et al. 2021; Carlsten et al. 2022, in yellow stars). The same relations are colour coded using the global g − r colour of the galaxy in the middle panels) and estimated age (see Sect. 4) in the lower panels. All the best-fit relations used and quoted in this work are provided in Table 3. The main features of these relations are:

Fig. 8. Size–stellar mass relation using edge radii (left) and the associated edge density–stellar mass relation (right). For visualisation reasons, we only plot the combined uncertainties due to the background and stellar mass estimation in our measurements (see text for details) for the field and Fornax LTGs/ETGs samples in the upper panels to demonstrate the typical values across the full stellar mass range of the scaling relations. Upper: Best-fit laws fitting the labelled dwarf samples: $R_{\mathrm{edge}}\sim M_{\star }^{1/3}$ (reported for galaxies with M⋆ > 107 M⊙ in Trujillo et al. 2020; Chamba et al. 2022) and $R_{\mathrm{edge}}\sim M_{\star }^{0.42}$ (this work). We plot the Fornax cluster dwarfs (purple pentagons), SAGA (Mao et al. 2021) and ELVES (Carlsten et al. 2022) satellites (yellow stars) and sub-sample Chamba et al. (2022) and Karachentsev et al. (2013) field galaxies (blue diamonds). For completeness, the radii for Fornax cluster massive late-type and early-type galaxies (black circles) are also shown. The shaded regions show the intrinsic scatter of each relation (∼0.06 dex for edge radii). At a fixed stellar mass, the majority of Fornax galaxies are below the best-fit line. The edge density-stellar mass plane is plotted using the same symbols and colours as in the left panels. The piecewise, Σ⋆, edge best-fit relations as well as the luminous ETGs and LTGs from Chamba et al. (2022) (grey crosses) are shown for reference. Middle: The same relations are now colour coded according to the global g − r colour of the galaxy. The symbols representing each sub-sample are those used in the upper panels. Lower: Similar to the middle panels, but now using the g − r colour to estimate the age of the galaxy from MILES (Vazdekis et al. 2012) for a fixed metallicity of [M/H] = –0.71 (see text for details). For reference, the age of the dwarf elliptical galaxies with M⋆ > 107 M⊙ in the Fornax Cluster is ∼10 Gyr (Rakos et al. 2001). The figure shows that galaxy edges in the Fornax cluster are smaller, denser and redder compared to the satellite and field samples. We use the best-fit relations to quantify and represent these results as histograms in Fig. 9.

Edge radii-mass. First, all the dwarfs with stellar masses < 10^7.5 M_⊙ collectively deviate from the one-third slope reported in Trujillo et al. (2020) and Chamba et al. (2022) for field galaxies. We find that the galaxies follow a size–mass relation with a higher slope of 0.42 ± 0.01.

Second, the observed scatter in the relation is 0.13 dex across the full mass range analysed. After accounting for the uncertainties due to background subtraction and stellar mass estimation as listed in Table 3, the dispersion of the relation reduces to 0.068 dex. Additionally removing the typical uncertainty from repeated identifications found in previous work (∼0.04 dex; Chamba et al. 2022) lowers this value to ∼0.055 dex, consistent with the findings from Trujillo et al. (2020).

Third, on average, the Fornax dwarfs are 0.04 dex below the mean edge–mass relation, while the rest of the sample is 0.05 dex above the relation. This deviation is highlighted in the histograms plotted in upper and middle panels of Fig. 9. The lower panel separates the total sample in three bins of the global g − r colour of the galaxies: g − r < 0.3, 0.3 < g − r < 0.5 and g − r > 0.5.

Fig. 9. Distribution of edge radii (left) and surface stellar mass densities (right) normalised by the expected value derived from the regression of these measurements against stellar mass ($R_{\mathrm{fit}}\sim M_{\star }^{0.4}$ and ⟨Σ⋆, edge⟩ from Chamba et al. (2022), respectively; see Fig. 8). In the upper and middle panels, Fornax dwarf galaxies are shown in purple. The individual samples considered are separated by the labelled designs (upper). The distribution of the rest of the sample (field + satellites combined) is plotted in black dotted lines. In the middle panel, we combine the field (Chamba et al. 2022 and Karachentsev et al. 2013 in blue) and satellite (SAGA; Mao et al. 2021 and ELVES; Carlsten et al. 2022 in yellow) samples to further visualise the distinction between their distributions. The lower panels show the distribution of galaxies in the size– and edge density–stellar mass planes when separated in three bins of g − r colour, showing that redder, older galaxies populate the lower half of the size–mass relation (left) and upper half in the density-stellar mass relation (right). Quantitatively, the panels show that compared to field galaxies, Fornax cluster galaxies can be up to 50% smaller and on average, redder, older and two times denser at the edge.

Fig. 9 shows that Fornax galaxies are on average 20% smaller compared to galaxies outside a cluster. If we consider only the nearly isolated sample, then Fornax cluster galaxies are on average 25% smaller. When considering the group satellites, the Fornax dwarf galaxies are 13% smaller than this sample. Within a 1σ interval, Fornax galaxies can be up to 50–60% smaller in edge radius than those in the field (Fig. 9, middle panel). In this sense, the satellite galaxies have intermediate sizes compared to the Fornax cluster and nearly isolated galaxies.

Fourth, the lower panels of Figs. 8 and 9, show that at a fixed stellar mass, younger and bluer galaxies are larger. In Fig. 10, we plot the same relations as in Fig. 8 colour coded by morphology. We find that this colour bifurcation is a reflection of morphology. In other words, bluer, late-type galaxies are larger than the redder, early type galaxies. This result is similar to that found in Chamba et al. (2022) for the late-type spiral galaxies (Sect. 2) but here we have extended it to even lower mass galaxies M_⋆ < 10⁹ M_⊙.

Fig. 10. Scaling relations colour-coded by morphology. Early-type galaxies are coloured in red are smaller on average than the late-type galaxies in blue across the stellar mass range studied. The dotted and dashed lines are the same as those previously plotted in the upper panels of Fig. 8.

Fifth, the five Fornax massive ETGs studied here are below the best-fit M^1/3 line. All except one of the massive LTGs with M_⋆ > 10⁹ M_⊙ are also below this line. This result implies smaller sizes relative to the field sample studied in Chamba et al. (2022).

Edge surface stellar mass density-mass. In relation to the above results is the edge surface stellar mass density – stellar mass relation, first published in Chamba et al. (2022). We plot the values measured in this work for the galaxies analysed here in the right panels of Fig. 8. The average edge density–mass relations from Chamba et al. (2022) for their field sample as well as the measurements for the more massive, luminous galaxies are over-plotted for reference. The same colour and age bifurcation we report in the edge radii–stellar mass plane is also observed in the edge surface stellar mass density-stellar mass relations (middle panels of Fig. 8).

We find that the majority of Fornax and satellite galaxies populate the upper half of the field relations. This region in the edge density–stellar mass plane systematically corresponds to redder, older and higher edge densities at a fixed stellar mass compared to the average values found in nearly isolated environments. In other words, the edges of cluster and satellite galaxies are denser compared to those found to be near isolated. On average, the edges of the Fornax (satellites) galaxies are two (1.5) times denser than in isolation.

Most of the galaxies with smaller sizes and higher edge densities in Fornax are of early-type morphology (Fig. 10.) Our results are therefore a reflection of the different edge properties found in early- and late-type galaxies. However, we further investigate this issue in Sect. 5.3 and demonstrate that the edge radii of late-type galaxies are also impacted by the environment.

The edge densities of the more massive Fornax galaxies are higher than ∼3 M_⊙/pc² in many cases. If we use the best-fit edge density-stellar mass relations from Chamba et al. (2022), the edge density values of the massive Fornax ETGs and LTGs is approximately a factor of four times higher than the mean edge density values of the C22 field galaxies with the same stellar mass. While we cannot perform a statistical analysis on the massive end of our sample in this work, these results suggest that the observation of higher edge densities in the Fornax Cluster occurs across the full mass range considered here.

Effective radii-mass. For comparison with the above results, we show the well-known effective radii-stellar mass scaling relation in Fig. 11. The upper panel colour codes the data points according to the environment and the lower panel according to our early- and late-type categories as in Fig. 10. As before, the best-fit relations are also plotted with the 0.2 dex scatter shaded for reference.

Fig. 11. Effective radius–stellar mass plane for galaxies in different environments. In the top panel, the data points are labelled as in Fig. 8. The lower panel is similar to Fig. 10. We colour-coded the effective radius–stellar mass relation, using red for the early-type and blue for the late-type galaxies. In both panels, the shaded regions correspond to the 0.2 dex scatter for reference.

First, the slope of the r_e–mass plane considering only the field sample is 0.25 ± 0.01 while with all the dwarfs it is 0.28 ± 0.01. These values are very similar, with the slope slightly higher when the satellites and Fornax galaxies are included. However, the scatter is double that of the edge–mass relation.

Second, there is no clear separation of the sub-samples in effective radius–mass plane. We verify this point by using the histograms shown in Fig. 12. This figure shows the distribution of radii at a fixed stellar mass for individual sub-samples and combined samples as in Fig. 9. The result further emphasises the fact that the effective radius has previously led to ambiguous results on the impact of the environment on sizes.

Fig. 12. Distribution of half-light radii, normalised by the expected value derived from the regression of re against stellar mass ($r_{\mathrm{fit}}\propto M_{\star }^{0.28}$; see Fig. 11). As in Fig. 9, Fornax dwarf galaxies are shown in purple. The upper panels shows the distribution of individual samples separated by the labelled designs as in the left of Fig. 9. The lower panel shows the full satellites (yellow) and the combined nearly isolated + satellites samples (blue). The total field sample is plotted in dotted black lines. There is no obvious distinction between the samples in the effective radius–stellar mass plane.

Third, although with a larger scatter, similar to the edge radii, almost all the more massive sub-sample of Fornax LTGs and ETGs studied here are below the best-fit lines, which implies smaller effective radii relative to the field sample studied in Chamba et al. (2022).

5.3. Comparison by morphology and environment

We reaffirm the above results in Figs. 13, 14 and 15 where we plot the median radii in bins of stellar mass to further analyse the differences in our sample as a function of their environment and morphology. In all figures, the median radii are computed in bins of stellar mass with bin size 0.5 dex and the plotted error bars are the 3σ clipped standard error. Our results do not change if we consider three or five σ clipped median and error values. The grey shaded region in each panel corresponds to the stellar mass regime where we lose statistics due to the low number of massive galaxies in the satellites and Fornax Cluster samples (see Fig. 4 and associated text).

Fig. 13. Median effective (upper) and edge radii (lower) in bins of stellar mass. The distribution of the number of galaxies used to compute the median radii in each stellar mass bin was plotted in Fig. 4. The shaded grey region corresponds to the stellar mass regime beyond which we lack cluster and satellite galaxies for a meaningful, statistical comparison. However, we compute the median points with the few galaxies in our sample for completeness. The separation between the Fornax cluster and field sample is much more clear in median edge radii: the cluster galaxies are systematically smaller in edge radii across the full stellar mass range considered, which is not reflected by the re measurements.

Fig. 14. Similar to Fig. 13: median radii for early- and late-type galaxies in our total sample. The error bars correspond to three times the dispersion in the estimated median value. Early-type galaxies are larger in effective radius when M⋆ ≲ 108 M⊙ compared to the late-type galaxies (upper). However, in their edge radii (lower), the early-type galaxies are systematically smaller by ∼20% than the late-type ones across the full stellar mass range.

Fig. 15. Similar to Fig. 13 but now separated by morphology and environment. The early-type galaxies are plotted on the left and late-type ones on the right. In both panels we plot those galaxies in the field by combining the nearly isolated and satellite samples (green squares). We also plot the combined line for the cluster using the full sample of early- and late-type galaxies (the same shown in Fig. 13) in the dot-dashed line. Both the early- and late-type galaxies are smaller in the Fornax cluster compared to the field. However, the early-type galaxies are even smaller than the late-type ones, regardless of the environment (Fig. 14).

In Fig. 13 we plot the median effective radius (upper panel) and the same for the edge radii (lower) for all the Fornax cluster galaxies (purple pentagons), group satellites (yellow stars) and isolated samples (blue diamonds). We find that when M_⋆ ≲ 10⁹ M_⊙, the median r_e of Fornax cluster galaxies are comparable to the isolated and satellite samples. This trend appears to switch at higher stellar masses where the median radii of Fornax galaxies are smaller. We point out, however, that our higher mass data points (grey shaded) suffer from low statistics (see Fig. 4). In Appendix C we show that our conclusions based on the use of the effective radii to study Fornax galaxies are independent of the method we used to derive them by comparing these relations with that derived from Su et al. (2021).

If the large scatter in the r_e − M_⋆ relation could be explained by differences in the sample environment, then it is reasonable to expect that the ratio between the median radii at a fixed stellar mass for different environmentally selected samples would have to be at least on the order of the intrinsic dispersion of the relation, namely 10^±σ_int, which is ∼0.72 for smaller galaxies below the best-fit scaling relation or ∼1.4 for larger galaxies above the line. We found that the average ratio between the median Fornax and isolated or satellite data points at each bin where M_⋆ ≲ 10⁹ M_⊙ is 0.9 − 1.0 ± 0.3. These values imply there are no significant differences in the radii of these galaxies.

In contrast, the median edge radii is systematically larger for the isolated and satellite samples compared to the Fornax cluster galaxies across the full stellar mass range. The average ratio between the data points at each bin where M_⋆ ≲ 10⁹ M_⊙ is 0.7 − 0.8 ± 0.12. If the scatter in the edge radii–stellar mass can be explained by the environmental differences in our sub-sample, then the expected difference in their radii using the intrinsic scatter of the relation is ∼0.86. Therefore, compared to the r_e, we can conclude that the edge radii–stellar mass relation is capable of separating the radii of galaxies in different environments at a fixed stellar mass.

To further analyse which galaxies are driving these trends by morphology, in Fig. 14, we separate our sample in to the early- and late-type categories and compute their median radii. While the early-type galaxies are consistently smaller than the late-type ones across stellar mass in their median R_edge, the trend is similar in effective radius only for galaxies with M_⋆ > 10^8.4 M_⊙. This trend reverses for lower mass galaxies as mentioned earlier.

Carlsten et al. (2021) have already analysed the effective radius–mass plane in detail in the same stellar mass range as in our work and their results are very similar (see their Figs. 5, 6 and 9 in particular). However, they point out that the galaxies in their stellar mass bin 10⁶ M_⊙ < M_⋆ < 10⁷ M_⊙ suffer from incompleteness in the surveys due to the limiting surface brightness of the images (their Fig. 2 and discussion therein). As we are using a sub-sample of their survey in this stellar mass regime, this argument also applies here. It is therefore currently difficult to interpret the difference in radii in our lowest stellar mass bins M_⋆ < 10⁷ M_⊙ physically as we are likely missing the faint, early- and late-type galaxies in our sample. Therefore, we did not repeat the same analysis as in Carlsten et al. (2021) and instead focused on our edge radii measurements for galaxies.

We repeated the same analysis that was presented in Chamba et al. (2022) Appendix B2 in order to examine the surface brightness near R_edge. We found that the surface brightness near the edges of the faintest galaxies in our low mass sample (μ_r(0)∼26 mag/arcsec² and M_⋆ < 10⁷ M_⊙) approach the limiting depth of the images. This result further supports our choice for leaving the analysis of edges in LSB galaxies using future deeper datasets (see Sect. 3). However, given that we only found five LSB-type galaxies in our sample that have size measurements likely impacted by the surface brightness limit, we did not remove these galaxies in our analysis, as doing so would not change our main results.

Nevertheless, in the median edge radii panels, we found that the early-type line is driven by our cluster sample where M_⋆ < 10^9.5 M_⊙. This point is more clearly visible in Fig. 15, where we further separated the early- and late-type galaxies according to their environment. We plotted the cluster line from the lower panel of Fig. 13, which includes both the early- and late-type samples for comparison. While the late-type cluster galaxies are larger than the early-type ones, these galaxies are still systematically smaller than those found in the isolated environment. The average ratio between the median Fornax and isolated data points at each bin for both the early- and late-type galaxies are 0.8 ± 0.1, that is, a difference of about ∼20%. However, both the cluster early- and late-type galaxies are closer to the satellite galaxies, with a ratio 0.9 ± 0.1.

Additionally, we plotted the median lines after grouping the satellite and isolated early- (left) and late-type (right) samples as ‘field’. Both the early- and late-type galaxies are smaller in the cluster compared to the field. The average ratio between the cluster and field median radii for both the early- and late-type samples are very similar ∼0.8 ± 0.1. However, we caution the reader that our sample in the field lacks early-type nearly isolated galaxies with M_⋆ < 10^7.5 M_⊙. Nevertheless, we may conclude that the median edge radii at fixed stellar mass for both the early- and late-type galaxies depends on the environment.

In other words, we find that both the early- and late-type galaxies in the cluster environment are systematically smaller compared to similar galaxies in the field. However, the early-type galaxies are generally even smaller compared to the late-type galaxies in both the cluster and the field. These results complement those we report at fixed stellar mass without considering the differences in morphology. The intrinsic scatter in the edge radii–stellar mass plane is thus driven by both morphology and environment.

6. Discussion

Our results indicate that the edges of Fornax cluster galaxies can be up to 50% smaller and denser compared to field galaxies. Additionally, at a fixed stellar mass in the edge-mass relation, younger and bluer galaxies were found to be larger than older and redder ones. This colour bifurcation is related to morphology where the bluer, late-type galaxies in our sample are larger than the redder, early type ones, similar to the previous results found in Chamba et al. (2022). The interpretation of these findings are discussed in Sects. 6.1 and 6.3, considering the physical processes that are dominant in the Fornax Cluster and other environments.

6.1. Physical processes impacting the formation of edges

6.1.1. Fornax Cluster

In a crowded cluster environment, three main processes may impact the evolution of galaxies. As galaxies move through the cluster, their gas can be removed due to the ram pressure from the hot intra-cluster medium (Gunn & Gott 1972), their stellar and gaseous material may be removed due to tidal forces from multiple galaxy-galaxy interactions and collisions with surrounding galaxies (Dressler 1984; Moore et al. 1996) or from the gravitational potential of the full cluster (Gnedin 2003). We explore the impact of each of these processes on galaxy edge properties in the following paragraphs.

There are several observational signatures of ram pressure and tidal forces acting on galaxies. In the case of ram pressure, the signatures include: (1) the observation of relatively gas-poor galaxies in clusters and groups compared to those in isolated environments (2) extended neutral or molecular hydrogen (HI or H₂, respectively) tails, together with their alignment with respect to (3) the presence of hot gas or halos traced with X-rays. In the case of tidal forces, observable signatures include the visual deformation of galaxy shapes (e.g. the ‘S’ shape, dislocated centers), stellar tidal structures in the outskirts or the dislocation of HI or H₂ with respect to the optical centres of galaxies (similar to ram pressure candidates) because H₂ is more strongly bound to the galaxy and thus require stronger forces for them to be removed.

All of the above signatures have been reported in the Fornax Cluster using optical imaging from FDS (Venhola et al. 2018, 2022), X-rays using the RÖntgen SATellite (Paolillo et al. 2002) or NASA’s Chandra X-ray Observatory (Scharf et al. 2005) and observations of molecular (Zabel et al. 2019; Morokuma-Matsui et al. 2022) and neutral hydrogen gas (Loni et al. 2021; Serra et al. 2023) with the Atacama Large Millimeter/submillimeter Array (ALMA) and Meer-Karoo Array Telescope (MeerKAT), respectively.

However, Asencio et al. (2022)’s recent, detailed MCMC and N-body analysis of the susceptibility of tidal forces acting specifically on the FDS dwarf galaxies in the Fornax Cluster provides further clarity on which of these processes might be shaping the optical morphology of the dwarfs. In particular, these authors compare the susceptibility of tidal forces from galaxy-galaxy interactions and from the cluster potential and show that the latter process is the more likely contributor for the visual deformations reported in the morphology of these galaxies by Venhola et al. (2018, 2022). By defining tidal susceptibility from the cluster potential as the ratio between the half-mass radius and the tidal radius of the dwarf (r_tidal ∝ M^1/3 where M is the total mass of the galaxy), these authors found that higher suspecibility values correspond to more disturbed visual morphologies.

Galaxy-galaxy encounters were shown to be irrelevant because the disruption time-scale of a dwarf after an interaction with a massive elliptical galaxy is much larger than the age of the dwarf galaxies themselves in the Fornax cluster (∼10 Gyr; Rakos et al. 2001). A more recent analysis of Fornax-like clusters in the Illustris simulations (Anetjärvi, M. et al. 2023⁶) also demonstrates that the influence of dwarf-cluster potential tidal forces is stronger relative to that from dwarf-other galaxy interactions. The latter are only more or similarly significant to the former on the order of once every 10 Gyr per dwarf, further supporting the findings in Asencio et al. (2022).

The removal of stars via tidal forces. To consider the impact of cluster tides on our sample, we used the published Venhola et al. (2022) classifications to check for visual deformations or signs of tidal features in the outskirts that could lead to the removal of stars. We find that the Fornax sample studied here are 81% ‘regular’ or ‘undisturbed’ (i.e. galaxies without obvious distortions or tidal tails), 15% ‘slightly disturbed’ (hints of irregular features in the outskirts) and only 2% each of class ‘disturbed’ (very clear tidal features or distortions) and ‘unclear’ (difficult to classify due to surrounding contaminants such as neighbouring sources or bright objects). The higher fraction of ‘regular’ shaped or ‘undisturbed’ galaxies is understood because we visually verify that our assumption of elliptical symmetry is an average representation of the data when deriving the radial profiles to identify edges. Such galaxies are consistently more regular in shape (but see Appendix B for the more massive asymmetric cases in Raj et al. 2019).

Given these results, it is reasonable to conclude that the removal of stars via tidal forces may not be the main physical process causing the Fornax edges to be smaller and denser. However, we cannot rule out the possibility that not all stellar tidal features are visible in FDS, as for example, recently demonstrated in the study of the visibility of tidal features and debris across a range of environments and stellar masses in Rubin-LSST mock imaging from NewHorizon (Martin et al. 2022) or the FIRE hydrodynamical simulation study of disrupting satellite galaxies (Shipp et al. 2023). Nevertheless, from current optical observations of the sample used here, tidal forces from both the cluster and galaxy interactions cannot be directly responsible for the edge properties reported here.

Moreover, this conclusion is further supported if we consider the expected impact of mass loss (including dark matter) in a galaxy, regardless of the process that caused it. When mass is lost, the depth of the potential well of the galaxy is decreased. Assuming that the velocity dispersion of the stars (1) remains unchanged or (2) increased by adding kinetic energy through the interactions, then stars will move at the same speed or faster after the mass is removed. In either case, stars will move into higher orbits because the potential well is shallower (Binney & Tremaine 2008). Therefore, we would expect a reduction of the density of stars in the outskirts. However, we find higher edge densities, further supporting our conclusion that mass loss cannot be the process responsible for our results. We discuss how this conclusion compares with that from previous work using the effective radii or fixed iso-density measures in Sect. 6.3.

Ram pressure takes it all. To consider the influence of ram pressure in edge properties, we compile the known total HI masses of the galaxies in our sample from the catalogues published in Karachentsev et al. (2013), Durbala et al. (2020), Zhu & Putman (2023) and Kleiner et al. (2023). We plot the ratio between these HI masses and the stellar masses computed here, that is, the HI fraction as a function of edge density in Fig. 16. The standard relations between HI and its ratio with stellar mass as a function of stellar mass (e.g. Papastergis et al. 2012; Maddox et al. 2015) for comparison with ALFALFA data (Durbala et al. 2020) are shown in Appendix E. From the samples studied here, Fig. 16 shows that galaxies with higher edge densities are those with lower HI-to-stellar masses ratios (r = −0.23).

Fig. 16. Ratio of HI-to-stellar masses of a sub-sample of local volume field galaxies (blue diamonds; 64 studied here out of the 96 with available DeCaLs imaging from Karachentsev et al. 2013); nine SAGA and 15 ELVES satellites (yellow stars; matched with the Durbala et al. (2020) and Zhu & Putman (2023) measurements, respectively); and the Fornax galaxies (purple pentagons; 17 studied here out of the 39 HI masses or upper limits reported in Kleiner et al. 2023) as a function of edge surface stellar mass density. The HI masses of the Fornax LTGs and ETGs are taken from Loni et al. (2021) when available (dark blue points). Higher edge densities are found when the H I-to-stellar masses ratios are lower. A best-fit to the data points is shown as a dashed line to highlight this trend.

This result can be connected with the recent finding that most of the Fornax dwarf galaxies are HI deficient (Kleiner et al. 2023). This deficiency is thought to have been caused by the removal of HI via tidal forces and ram pressure. Tidal forces are required to explain their observations for two main reasons. The first is that ∼ half of the HI detected dwarfs have asymmetric stellar bodies, which is a signature of tidal forces acting on these galaxies.

The second reason is that all the detected HI rich dwarfs ‘avoid’ the highly dense regions of the cluster. The densest regions are those populated by the most massive potential wells in the core. The lack of HI rich dwarfs in those regions imply that the dwarfs in the core (see Fig. 3) have completely lost their HI in the cluster.

Ram pressure is needed to explain the observations mainly because of the configuration of the detected HI tidal tails in these dwarfs. As ram pressure depends on the density of the intracluster medium (Gunn & Gott 1972), the tails are shaped by the X-ray emitting sources in the more dense core of the cluster (Scharf et al. 2005, see their Fig. 12).

Therefore, while we have established that the removal of stars via tidal forces cannot result in high density edges, the forces can still remove the gas component in a galaxy. Considering that the distribution of HI is several times more extended than the optical counterpart of galaxies (e.g. Wang et al. 2016), the detection of HI tidal tails suggests that the HI in the outskirts is likely removed first. Initially, the removal occurs via tidal forces and subsequently ram pressure may continue to remove the loosely bound atomic or molecular gas. This removal may prevent further inflow of cold gas and lowers the respective gas densities.

The lowering of both HI and H₂ gas density as these components are removed has indeed been reported in the Watts et al. (2023) study of the radial stellar and gas (HI and H₂) density distributions of Virgo cluster galaxies. These authors find that while the impact on HI is considerably stronger than molecular gas, the H₂ density can be reduced by a factor of three in the cluster galaxies. They additionally show that it is the lowest density molecular gas, preferentially found in low stellar density regions (i.e. larger radii), that is removed first (see their Fig. 7).

This finding is consistent with earlier work on the comparison between the optical R-band radial distribution and that of a star formation rate indicator, H_α, in Virgo galaxies by Koopmann & Kenney (2004). These authors report the existence of extremely truncated galaxies in Virgo where there is no star formation in the outer 60–70% regions of their disks (Koopmann et al. 2001). In a later study considering a larger sample, (Koopmann & Kenney 2004) show that the star formation rate in these outer regions has in fact been reduced by a median factor of 9 compared to more isolated galaxies. In other words, star formation is truncated outside-in (Bekki 2009; Zhang et al. 2012).

Although Fornax is a lower mass cluster compared to Virgo and at a different stage of evolution, there are several pieces of evidence that support the idea that a similar phenomenon might be found in Fornax. For instance, when H₂ deficiency is defined as the difference between the observed H₂ mass and the statistically expected H₂ mass from galaxies of the same stellar mass in the field, Zabel et al. (2019) has shown that the H₂ deficiencies of Fornax galaxies are even higher than in Virgo (down to ∼–1.1 dex compared to ∼–0.3 dex). The higher reduction of HI compared to H₂ in Fornax has also been reported in Morokuma-Matsui et al. (2022). Therefore, we can already anticipate that lower gas densities will also be removed first in Fornax cluster galaxies.

Morokuma-Matsui et al. (2022) additionally report that the lower cold gas fractions (defined as the total HI and H₂ mass as a fraction of stellar mass) due to gas removal is the main cause for the lower star formation activity in Fornax galaxies. If the edges we identify are indeed related to a gas density threshold needed for star formation, then lower gas densities in the outskirts will prevent the formation of stars in those regions if they do not meet the necessary threshold, thus truncating star formation outside-in (see e.g. Bekki 2009).

Our finding that Fornax galaxy edges are denser can thus be understood with the scenario that the removal of low density H I and molecular gas via cluster tides and ram pressure exerted on these galaxies during in-fall or as they cross the cluster likely truncated star formation outside-in, preventing the formation of lower density edges and larger sizes. Given that not all galaxy clusters are at the same stage of assembly, have the same mass or interstellar medium (for example the density of hot halos in the Virgo cluster is a factor ∼4 higher than in Fornax; see the comaprison in Zabel et al. 2019) it would be interesting to explore this scenario in other clusters.

6.1.2. Group satellite galaxies

Satellites of massive galaxies may also be influenced by tidal forces and ram pressure. Peñarrubia et al. (2009) for example study how the shape of the radial profile changes as satellite dwarfs lose stars from tidal forces. They show that as galaxies lose stars, their outer stellar surface density radial profile up-bends, and eventually approaches a R^1/4 law. By applying their model to observations of Local Group satellites, these authors provide the predictions for the location of the ‘break’ radii in these galaxies.

A ‘break’ in Peñarrubia et al. (2009) is defined as the location where the profile bends upward because of tidal forces. The authors show that the predicted ‘break’ radii from these models are well beyond the observed breaks in all of the Local Group dwarf galaxies studied except the Sagittarius dwarf, a galaxy with very clear tidal perturbations.

In fact, the predicted radii are also well beyond the edges we find here for satellite galaxies of similar stellar mass. We find that the predicted break radii from Peñarrubia et al. (2009) are located at ∼2 − 35 R_edge depending on the stellar mass of the dwarf. These results indicate that the majority of edges we find here are likely not of tidal origin, similar to the case of Fornax cluster dwarfs.

The comparison of the shapes of profiles themselves is beyond the scope of our work and is highly complicated, as it requires the deconvolution of the PSF. Locating the edges themselves does not require this step (see Fig. 12 in Trujillo & Fliri 2016 or Appendix C in Chamba et al. 2022). We leave such an analysis for when extended PSFs for DECam have been established.

More recently, Putman et al. (2021) have analysed the impact of the gaseous halo medium of the Milky Way and Andromeda on Local Group dwarfs to study the role of ram pressure and its responsibility in the removal of gas in satellites. These authors show that the minimum halo density to completely remove the gas in the satellites via ram pressure is reached well within the distances to their respective hosts (whichever one is closer). This finding explains the lack of gas-rich galaxies in the Local Group within the virial radii of their hosts. Similar to our study, many of their galaxies do not show signs of tidal distortions or tails and they argue that the impact of tidal forces is likely negligible in comparison to ram pressure.

These results have been further supported by observations of satellite systems in nearby spiral galaxies. In the absence of H I measurements for all the ELVES and SAGA satellites in ALFALFA, Zhu & Putman (2023) used the late-type morphologies, the presence of H_α or both as an indicator for the presence of gas in these galaxies. Using ELVES and SAGA in combination with their own sample of satellite galaxies selected using ALFALFA, Zhu & Putman (2023) report a similarly low number of gas rich satellites as that in the Local Group. The authors find ∼0 − 3 gas-rich satellites within the virial radius of the host galaxy (see their Fig. 11). Given this similarity, they argue that ram pressure from the host medium can also explain the lack of gas-rich satellites hosted by Milky Way analogues in the nearby Universe as in the Local Group studied by Putman et al. (2021) (see also Greene et al. 2023).

Our results that the ELVES and SAGA satellites also have high density edges and smaller sizes compared to the nearly isolated sample is broadly consistent with these findings. In fact, the satellites studied here seem to populate an intermediate regime in terms of size and edge density (Fig. 9) between the Fornax and the field samples. This result potentially indicates that the impact of ram pressure alone from a single host halo on edge properties is weaker than the cluster tidal+ram pressure scenario discussed above for the case of Fornax galaxies. It would be interesting to compare these results in future work with the predictions of hydrodynamical simulations that can resolve dwarf galaxies in low and high density environments.

6.1.3. Massive galaxies

While much of the discussion above has been specific to dwarf galaxies, here we make a few brief remarks on the edge properties of the more massive galaxies in our sample. Our final sample consists of only five massive ETGs from Iodice et al. (2019) and 12 massive LTGs from Raj et al. (2019, 2020) who studied their inner ‘break’ radii. For comparison, the break radii reported in Raj et al. (2019, 2020) are at 0.3 − 0.7 R_edge and at a compatible location only in one galaxy FCC013. There could be potential physical connections between the break and edge radii, and they may have a common origin, as suggested in Martínez-Lombilla et al. (2019). However, exploring this connection is beyond the scope of the current work.

Unfortunately, given the low number of galaxies we also cannot perform a statistical analysis in the stellar mass range of these LTGs and ETGs as we did for galaxies in the lower mass regime. Nevertheless, we point out a few obvious differences in the sample of galaxies studied here and their edge properties. Contrary to the dwarf galaxies in our sample, most of the Fornax LTGs studied here have signs of galaxy interactions (FCC number 312), have slightly asymmetric stellar discs (FCC 113, 121, 285) or transitioning to S0 type (FCC 290).

As discussed in Raj et al. (2019, 2020), minor mergers or a recent fly-by galaxy-galaxy interaction may impact the stellar distribution of these galaxies by truncating their star formation outside-in due to the removal of gas. While we cannot use the edge properties to pinpoint the exact formation channel, we find that the densities of the edges are a factor of four times higher compared to the mean expectation from galaxies of similar stellar mass in the field. This result is indicative that galaxy interactions may have a much stronger impact on the properties of edges than cluster tides or ram pressure alone.

Many of the Fornax A sub-group galaxies are also recent or intermediate infallers (FCC numbers 013, 029, 033, 035, 062 from the galaxies in our sample; see the phase space diagram Fig. 7 in Raj et al. 2020). While we do not report any significant differences between the edge properties of infallers and galaxies within the main cluster (see Sect. 6.3), there are clear differences between the edge properties of the LTGs and ETGs. The edge densities of ETGs are typically even higher than those of the LTGs by a factor ∼1.5 within the Fornax environment (this work), while in the field they are greater than a factor of three (Chamba et al. 2022). We discuss the differences between galaxy types in the next section.

6.2. The origin of scatter in the size–stellar mass plane

Differences in the physical properties between galaxy types is expected given that galaxies may undergo varied star formation histories, regardless of their present-day local environment. For instance, massive elliptical galaxies show highly efficient star formation at early times (e.g. Jaskot et al. 2015), in stark contrast to the more inefficient formation found in dwarfs (e.g. Huang et al. 2012). The median higher edge densities ∼5 M_⊙/pc² for the massive ETGs, ∼3 M_⊙/pc² for the massive LTGs and ∼1 M_⊙/pc² for the full dwarf sample we find in the Fornax Cluster is potentially reflective of this intrinsic difference. In the field, the values reported in Chamba et al. (2022) were > 3 M_⊙/pc² for massive ETGs, ∼1 M_⊙/pc² for massive LTGs and ∼0.6 M_⊙/pc² for dwarf galaxies.

Interestingly, by further separating the our sample in to early- and late-type dwarf galaxies, we have additionally shown that the early-type dwarfs are systematically smaller than those categorised at late-type in the Fornax cluster at a given stellar mass. Both dwarf types are also smaller in the Fornax cluster compared to similar galaxies in the field. These findings are potentially linked with the colour and age bifurcation we find in the edge scaling relations shown in Fig. 8.

Collectively, they suggest that Fornax early-type galaxies reached their present-day sizes earlier than the late-type galaxies. In fact, all but three of the gas-rich dwarfs in Fornax plotted in Fig. 16 are of late-type morphology. Most of the early-type dwarf galaxies likely lost their gas earlier and the cluster gas-rich galaxies are more recent infallers still evolving in the new environment (Kleiner et al. 2023). The colour, age and morphology bifurcations we find in the size–stellar mass relations are thus likely driven by the environment of galaxies. In other words, the scatter of the size–stellar mass plane can be physically explained by the differences in morphology and environment of galaxies. Future work using much larger samples of galaxies, including gas-rich early-type galaxies (e.g. Serra et al. 2012), are welcome to explore this issue in more detail and verify our findings.

6.3. The dependence of galaxy sizes on cluster-centric distance

As discussed in Venhola et al. (2019) in particular for the Fornax Cluster, the cluster-centric distance defined as the distance of the galaxies with respect to the brightest cluster galaxy NGC 1399, can be used as a robust proxy for galaxy environment (see their Fig. 4 showing its strong correlation with projected galaxy density). These authors find that for the brightest dwarfs, their effective radii decrease as a function of cluster-centric distance. However, the most diffuse, low surface brightness dwarfs with large effective radii are systematically found in the central regions of the cluster compared to those with smaller effective radii (Venhola et al. 2022).

However, in our work, we do not currently find an obvious dependence of the Fornax edge radii or densities on cluster-centric distance. This result does not change even if we compare the edge properties within the in-falling Fornax A sub-group and the main cluster, the locations of which are shown in Fig. 3. Our main explanation for this result is as follows.

For a dwarf galaxy with stellar mass ∼10⁸ M_⊙, the gas removal time scale is ∼240 Myr (Kleiner et al. 2023). For comparison, if we consider the simplest estimate of the cluster crossing time of a galaxy using a relative velocity dispersion of 370 km/s (Drinkwater et al. 2001b) and the virial radius of 700 kpc of Fornax, the crossing time scale is approximately 1 − 2 Gyr (Loni et al. 2021). These timescales suggest that gas removal is possible over an extended period that encompasses multiple cluster crossings (see Kleiner et al. 2023), implying that a dwarf could be subjected to tidal forces and ram pressure at different stages of its lifetime (e.g. Jaffé et al. 2018; Yun et al. 2019).

However, it is unknown at which location in the galaxy’s trajectory was the gas in the outskirts significantly removed or lowered in order to impact their size. In other words, the galaxy can reach its present-day size by undergoing the aforementioned cluster processes and still continue to cross the cluster. The galaxy’s present-day location may thus not necessarily reflect the location where its edge was most impacted.

The rapid quenching of dwarf galaxies due to gas removal in observations and simulations as soon as these galaxies enter the cluster environment has already been shown in earlier work (e.g. Boselli & Gavazzi 2014). After such quenching, the dwarfs may then not significantly change their size no matter the number of times they could cross the cluster or be impacted by ram pressure at their current location. As the outskirts are more vulnerable compared to the radii traced by the effective radius, it is then reasonable to expect that any dependence on distance can be more easily erased.

While the above argument could apply for most of the dwarfs in Fornax, we point out that our sample does not include the faintest, low surface brightness dwarfs studied in Venhola et al. (2022). Future work characterising the edges of such galaxies specifically may provide a more complete picture to the study performed here. Nevertheless, the lack of a dependence on cluster-centric distance we find here compared to previous works using the effective radii highlights that our edge measure is much more sensitive to local environmental processes possible all over the cluster region and is independent of inner structure or light concentration. In Appendix F we compare and discuss the use of other size indicators in previous work.

7. Conclusions

The impact of the environment on galaxy size has long been elusive. Previous work addressing this issue using the effective radius, a measure that traces light concentration, have ignored the fact that the definition is often biased against the outskirts of galaxies. Addressing this bias is crucial because the outskirts of galaxies harbour key signatures of environmental processes, such as ram pressure or galaxy-galaxy interactions, which may transform the surrounding interstellar medium and consequently the galaxy’s outer growth.

After several decades of research where the effective radius has shown inconclusive or ambiguous results, we demonstrate the significant impact of the environment on galaxy size when using a physically motivated size definition that is more representative of the in situ stellar boundaries, or edges, of galaxies (Trujillo et al. 2020; Chamba et al. 2020, 2022). Our main results are as follows:

• The edge radii–stellar mass relation for all galaxies within 10⁶ − 10¹⁰ M_⊙ is $R_{\mathrm{edge}}\sim M_{\star }^{0.42}$, with a very low intrinsic scatter ∼0.06 dex. Compared to the effective radius, the dispersion of this size–stellar mass plane is reduced by a factor of more than two.

• The scatter in the edge radii-stellar mass plane is physically driven by the morphology and environment of galaxies. Fornax cluster galaxies can have up to ∼50% smaller edge radii than those found in nearly isolated or field environments. The stellar surface density at these edges are 50% more dense compared to the field.

• When considering morphology, at a fixed stellar mass, the redder, older early-type galaxies in the cluster are smaller by ∼20% compared to the late-type galaxies across the full range studied. A similar difference between galaxy types is found in the field.

• Confirmed satellites follow similar trends when compared to the nearly isolated galaxies. However, their sizes are still larger than the cluster galaxies.

• Galaxies with lower HI fractions have edges with higher stellar surface density.

These results are consistent with recent HI observations of the Fornax Cluster, which show that this gas component in member galaxies has been rapidly removed as a consequence of cluster tides and ram pressure (Loni et al. 2021; Kleiner et al. 2023). These environmental processes truncate galaxies outside-in earlier than in the field, which implies that cluster galaxies may have also reached their present-day sizes earlier. In this scenario, the formation of larger sizes and low density edges in a cluster environment becomes more difficult.

Finally, our work highlights the importance of deep imaging surveys to characterise the outer growth of the faintest galaxies, a regime that would have otherwise remained invisible. Faint galaxies are not only important for understanding how environmental processes affect the dark matter content of galaxies (Jackson et al. 2021; Rodriguez et al. 2021; Mao et al. 2022; Asencio et al. 2022; Mau et al. 2022; Lucie-Smith et al. 2023; Montes et al. 2024) but also for the future interpretation of upcoming deep, multi-band observations from the Vera Rubin Observatory (Martin et al. 2022; Shipp et al. 2023; Tsiane et al., in prep.). Our findings thus offer a unique approach for the future detection of the low surface brightness imprints on galaxy evolution and environment.

Acknowledgments

We thank the anonymous referee for their comments, which improved the quality of our manuscript. This paper has undergone internal review in the LSST Dark Energy Science Collaboration. The internal reviewers were Jeff Carlin, Yuanyuan Zhang and Ian Dell’Antonio. This work used the catalogues kindly provided by Aku Venhola (FDS) and Yao-Yuan Mao (SAGA) and benefited from comments by Reynier Peletier, Maria Angela Raj, Aaron Watkins, Maria Carmen Toribio, Kelley Hess, Dane Kleiner and Amélie Saintonge. NC performed the analysis and wrote the paper. MH discussed the results, implications and commented on the manuscript at all stages. NC and MH acknowledge support from the research project grant ‘Understanding the Dynamic Universe’ funded by the Knut and Alice Wallenberg Foundation under Dnr KAW 2018.0067. The DESC acknowledges ongoing support from the Institut National de Physique Nucléaire et de Physique des Particules in France; the Science & Technology Facilities Council in the United Kingdom; and the Department of Energy, the National Science Foundation, and the LSST Corporation in the United States. DESC uses resources of the IN2P3 Computing Center (CC-IN2P3–Lyon/Villeurbanne – France) funded by the Centre National de la Recherche Scientifique; the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231; STFC DiRAC HPC Facilities, funded by UK BEIS National E-infrastructure capital grants; and the UK particle physics grid, supported by the GridPP Collaboration. This work was performed in part under DOE Contract DE-AC02-76SF00515. The FDS data were produced by the FDS collaboration and the data products are available via the https://archive.eso.org/scienceportal/home?publ_date=2020-08-26. See Peletier et al. (2020) for a full description of this data release. Funding for the Sloan Digital Sky Survey V has been provided by the Alfred P. Sloan Foundation, the Heising-Simons Foundation, the National Science Foundation, and the Participating Institutions. SDSS acknowledges support and resources from the Center for High-Performance Computing at the University of Utah. The SDSS web site is www.sdss.org. SDSS is managed by the Astrophysical Research Consortium for the Participating Institutions of the SDSS Collaboration, including the Carnegie Institution for Science, Chilean National Time Allocation Committee (CNTAC) ratified researchers, the Gotham Participation Group, Harvard University, Heidelberg University, The Johns Hopkins University, L’Ecole polytechnique fédérale de Lausanne (EPFL), Leibniz-Institut für Astrophysik Potsdam (AIP), Max-Planck-Institut für Astronomie (MPIA Heidelberg), Max-Planck-Institut für Extraterrestrische Physik (MPE), Nanjing University, National Astronomical Observatories of China (NAOC), New Mexico State University, The Ohio State University, Pennsylvania State University, Smithsonian Astrophysical Observatory, Space Telescope Science Institute (STScI), the Stellar Astrophysics Participation Group, Universidad Nacional Autónoma de México, University of Arizona, University of Colorado Boulder, University of Illinois at Urbana-Champaign, University of Toronto, University of Utah, University of Virginia, Yale University, and Yunnan University. The Legacy Surveys consist of three individual and complementary projects: the Dark Energy Camera Legacy Survey (DECaLS; Proposal ID 2014B-0404; PIs: David Schlegel and Arjun Dey), the Beijing-Arizona Sky Survey (BASS; NOAO Prop. ID 2015A-0801; PIs: Zhou Xu and Xiaohui Fan), and the Mayall z-band Legacy Survey (MzLS; Prop. ID 2016A-0453; PI: Arjun Dey). DECaLS, BASS and MzLS together include data obtained, respectively, at the Blanco telescope, Cerro Tololo Inter-American Observatory, NSF’s NOIRLab; the Bok telescope, Steward Observatory, University of Arizona; and the Mayall telescope, Kitt Peak National Observatory, NOIRLab. Pipeline processing and analyses of the data were supported by NOIRLab and the Lawrence Berkeley National Laboratory (LBNL). The Legacy Surveys project is honored to be permitted to conduct astronomical research on Iolkam Du’ag (Kitt Peak), a mountain with particular significance to the Tohono O’odham Nation. NOIRLab is operated by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation. LBNL is managed by the Regents of the University of California under contract to the U.S. Department of Energy. This project used data obtained with the Dark Energy Camera (DECam), which was constructed by the Dark Energy Survey (DES) collaboration. Funding for the DES Projects has been provided by the U.S. Department of Energy, the U.S. National Science Foundation, the Ministry of Science and Education of Spain, the Science and Technology Facilities Council of the United Kingdom, the Higher Education Funding Council for England, the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign, the Kavli Institute of Cosmological Physics at the University of Chicago, Center for Cosmology and Astro-Particle Physics at the Ohio State University, the Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University, Financiadora de Estudos e Projetos, Fundacao Carlos Chagas Filho de Amparo, Financiadora de Estudos e Projetos, Fundacao Carlos Chagas Filho de Amparo a Pesquisa do Estado do Rio de Janeiro, Conselho Nacional de Desenvolvimento Cientifico e Tecnologico and the Ministerio da Ciencia, Tecnologia e Inovacao, the Deutsche Forschungsgemeinschaft and the Collaborating Institutions in the Dark Energy Survey. The Collaborating Institutions are Argonne National Laboratory, the University of California at Santa Cruz, the University of Cambridge, Centro de Investigaciones Energeticas, Medioambientales y Tecnologicas-Madrid, the University of Chicago, University College London, the DES-Brazil Consortium, the University of Edinburgh, the Eidgenossische Technische Hochschule (ETH) Zurich, Fermi National Accelerator Laboratory, the University of Illinois at Urbana-Champaign, the Institut de Ciencies de l’Espai (IEEC/CSIC), the Institut de Fisica d’ltes Energies, Lawrence Berkeley National Laboratory, the Ludwig Maximilians Universitat Munchen and the associated Excellence Cluster Universe, the University of Michigan, NSF’s NOIRLab, the University of Nottingham, the Ohio State University, the University of Pennsylvania, the University of Portsmouth, SLAC National Accelerator Laboratory, Stanford University, the University of Sussex, and Texas A&M University. BASS is a key project of the Telescope Access Program (TAP), which has been funded by the National Astronomical Observatories of China, the Chinese Academy of Sciences (the Strategic Priority Research Program ‘The Emergence of Cosmological Structures’ Grant XDB09000000), and the Special Fund for Astronomy from the Ministry of Finance. The BASS is also supported by the External Cooperation Program of Chinese Academy of Sciences (Grant 114A11KYSB20160057), and Chinese National Natural Science Foundation (Grant 12120101003, 11433005). The Legacy Survey team makes use of data products from the Near-Earth Object Wide-field Infrared Survey Explorer (NEOWISE), which is a project of the Jet Propulsion Laboratory/California Institute of Technology. NEOWISE is funded by the National Aeronautics and Space Administration. The Legacy Surveys imaging of the DESI footprint is supported by the Director, Office of Science, Office of High Energy Physics of the U.S. Department of Energy under Contract No. DE-AC02-05CH1123, by the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility under the same contract; and by the U.S. National Science Foundation, Division of Astronomical Sciences under Contract No. AST-0950945 to NOAO. Software: Astropy, (http://www.astropy.org) a community-developed core Python package for Astronomy (Robitaille et al. 2013; Price-Whelan et al. 2018); SciPy (Jones et al. 2001); NumPy (Oliphant 2006; Walt 2011); Matplotlib (Hunter 2007); Jupyter Notebooks (Kluyver et al. 2016); TOPCAT (Taylor 2005); Sourcerer/MTObjects (Teeninga et al. 2016); Matrioska (mascara function; Borlaff et al. 2018) SWarp (Bertin 2010); and SAO Image DS9 (Smithsonian Astrophysical Observatory 2000).

References

Appendix A: Radial profile binning

To test the dependence of our edge identification on the bin size we use to derive radial profiles, we show the profiles of a bright dwarf galaxy in the Fornax cluster, FCC035, using different bin sizes in Fig. A. The legend indicates the bin size used to derive each profile in pixel units. For the FDS images, 1 pixel = 0.21 arcsec. All other steps are the same (see Sect. 4).

As expected, using larger bin sizes reduces the fluctuation between data points or smooths the profiles. This smoothing occurs because we are averaging flux over a larger area per elliptical annuli quantified by the bin size. Regardless of this smoothing effect, the location of the edge feature (vertical dashed line) in the g − r colour profile does not change.

Fig. A.1. Radial profiles of the dwarf galaxy FCC035 using different bin sizes. The bin sizes are in pixel units. While increasing the bin size reduces the noise in the profiles as expected, the shape of the profiles and the feature we identify as the location of the edge do not change.

Appendix B: Profiles of asymmetric galaxies

As we discuss in Sect. 6, the LTG sample from Raj et al. (2019) consists of a few asymmetric galaxies. We did not develop any special ad hoc methods to deal with these galaxies and used the same profile and edge derivation method as described in Sect. 4. We show the profiles of four of the extreme cases in Fig. B.1.

Fig. B.1. Radial profiles of asymmetric galaxies from the Raj et al. (2019) LTG sample. These galaxies are within the virial radius of the main Fornax cluster and are clearly asymmetric in some form (see text for details). In all cases, our default radial profile derivation permits the identification of an edge feature in the colour or density profiles (vertical dashed line). However, as we are using fixed elliptical annuli, our profiles should be considered as an average representation of the data.

All these galaxies are within the virial radius of the main cluster. From the upper to lower panels, FCC113 is a galaxy with a displaced central region, FCC285 is a jellyfish-like galaxy, FCC312 has large tidal features in its outer region and FCC121 is more truncated on one side of its disk. In all cases, our default radial profile derivation permits the identification of an edge feature in the colour or density profiles (vertical dashed line). However, we remark that it would be interesting in future work to explore how changing the axis ratio in the profile derivation step would impact the identification of the edge using model galaxies. We did not work on this idea for our sample, as we did not want to report edges that could be artificially caused by a change in position angle or axis ratio (see Sect. 4). But this does not mean that the elliptical parameters could change or be different near the location of the edge. Therefore, for these galaxies our profiles should be considered as an average measurement.

We also point out that in the case of FCC312, which has large tidal features in its outer part, the feature we identify as the edge is capable of separating the main body of the galaxy (mainly in situ) and its tidal feature (predominantly ex situ). This idea was previously shown in Trujillo et al. (2020) (see their Fig. 3) and used to define the stellar halo (see Trujillo et al. 2021). Future work analysing the stellar halo component of massive galaxies may consider using the edge as a physically motivated approach to define these structures.

Appendix C: Using the effective radii from Su et al. (2021)

To confirm that the main conclusions of our paper regarding the use of the effective radius are independent of the method we used to compute the radii, we plot the effective radii–stellar mass scaling relation using the Su et al. (2021) modelled values for Fornax galaxies in the upper panel of Fig. C.1. The middle and lower panels are as in those shown in Fig. 12 and 13 and only show slight differences.

Fig. C.1. Reproducing the scaling relations and histograms in Fig. 11, 12, and 13 using the modelled effective radii values from Su et al. (2021) for Fornax galaxies. Our main conclusion that the effective radii of Fornax galaxies are not significantly different from those found in the field for low stellar mass galaxies < 109 M⊙ remains unchanged.

Compared to the upper panel of 13, the median radii value in the lowest stellar mass bin is slightly lower and higher in the highest mass bins, although the latter data point suffers from poor statistics as mentioned earlier. The best-fit slope of the relation using the Su et al. (2021) values is slightly higher (0.29±0.02) but well within the uncertainties of the fit using our measurements (see Table 3). Despite these slight differences, our main conclusion that the effective radii of Fornax galaxies are not significantly different from those found in the field for low stellar mass galaxies < 10⁹ M_⊙ remains unchanged.

Appendix D: Alternative metallicity values in the size–stellar mass relation

In Fig. 8, we use a fixed low metallicity value [M/H] = -0.71 taken from the E-MILES models to estimate the ages of galaxies, accounting for the stellar populations in their outskirts. As an alternative estimation method, we can consider the Kirby et al. (2013) average stellar mass-metallicity relation for local dwarf galaxies where [M/H] $\sim M_{\star }^{0.3}$ if we assume solar abundance ratios. The scatter in the Kirby et al. (2013) relation is ∼ 0.17 dex. While the Kirby et al. (2013) relation strictly holds for galaxies < 10⁹ M_⊙, their Fig. 9 shows that we can extend the relation to galaxies with M_⋆ < 10¹⁰ M⊙ as the average relation for these higher mass galaxies shown in previous work is still within the scatter of the Kirby et al. (2013) extrapolation (see a more recent study by Domínguez-Gómez et al. 2023).

If we assume the Kirby et al. (2013) relation, which is applicable for 10⁶ M_⊙ < M_⋆ < 10¹⁰ M_⊙, then the relevant stellar metalicities are -1.71, -1.31 and -0.71. We show the relations using these values below in Fig. D.1. While the differences between the relations using -1.71 and -1.31 are minor, there are larger differences in individual points with the -0.71 relation we presented previously in Fig. 8. The ratio between the age estimated using either -1.71 or -1.31 and -0.71 deviate by more than a factor of 1.5 for half our sample. But given that we are interested in relative age distributions, the bifurcation we report in this work remains unchanged when considering any of these relations separately.

Fig. D.1. Same size–stellar mass relations as in Fig. 8 but colour-coded according to their age as estimated using the g − r colour at the fixed metallicity value stated in the colour bar. While the differences between the relations using -1.71 and -1.31 are minor, there are larger differences in individual points with the -0.71 relation. However, given that we are interested in relative age distributions, the bifurcation we report in this work remains unchanged when considering either of those relations separately (see text for discussion).

Further examination using the Kirby et al. (2013) relation suggests that the average metallicity value at our median stellar mass ∼10⁸ M_⊙ is closer to -1.00 and could likely be more representative of our sample. Unfortunately, as the E-MILES predictions for SDSS filters are not already available at this metallicity value, we do not currently have the models to present them here. Therefore, we adopted the same value as in Chamba et al. (2022), that is, -0.71. However, we point out that using the alternative metallicity value close to -1.00 (i.e. -1.31) would only strengthen the age and colour bifurcation we report in this work. We conclude that using either value would thus not change our results significantly.

Appendix E: H I properties and scaling relations

In this section, we show that our result from Fig. 16 where galaxies with lower HI fractions possess higher edge stellar densities, is likely not a selection effect. Complementing Fig. 16 is Fig. E.1 where we plot the HI-stellar mass relation in the upper panel (as in Maddox et al. 2015) and the HI-stellar mass ratio as a function of stellar mass (Papastergis et al. 2012) in the lower panel for a sub-sample of galaxies in our work as labelled. The ALFALFA detections for SDSS galaxies from Durbala et al. (2020) as well as Arecibo Observatory and Green Bank Telescope targeted observations of SDSS dwarfs by Geha et al. (2006) are also included for reference.

Fig. E.1. HI-stellar mass relation (upper) and the HI-stellar mass ratio as a function of stellar mass (lower) for a sub-sample of Fornax, satellites, and field galaxies in our work as labelled. Arecibo Observatory and Green Bank Telescope targeted observations by Geha et al. (2006) of optically selected dwarfs and ALFALFA detections (both for SDSS galaxies) from Durbala et al. (2020) are also included for reference. The offsets between the datapoints in our sub-samples, the Geha et al. (2006) work and ALFALFA is expected when galaxies are optically selected (see text for details). Arrows show HI upper limits where reported in the respective catalogues.

In the HI-stellar mass plane, our low mass galaxies populate the lower M_⋆ < 10⁹ M_⊙ end of the ALFALFA+SDSS sample. As discussed in Maddox et al. (2015), this regime is generally populated by low mass, low metallicity, irregular galaxies that are HI-rich. And the few that lie above M_⋆ > 10⁹ M_⊙ are in the regime populated by extended HI disks and massive galaxies where star formation is efficient. The bending or break of the relation at a limiting HI fraction is currently thought to be set by an upper limit in the spin parameter of the halo (Maddox et al. 2015). While we do not attempt to estimate the spin of the galaxies in our sample, the fact that the most H I poor galaxies in our work (Fornax) lie even below this relation in both regimes is expected because ALFALFA consistently detects the brighter HI-rich galaxies given the shallower depth of the data compared to MeerKAT observations for Fornax.

The systematic offset between the ALFALFA and all our other sub-sample datapoints is also seen in the lower panel. As discussed and also shown by Papastergis et al. (2012, in their Fig. 19), because ALFALFA is a blind HI survey where the integrated time of observations per pointing is fixed, the distribution of ALFALFA detections in the HI-stellar mass ratio vs. stellar mass plane is expected to lie above of those from targeted HI observations of optically selected galaxies. We reproduce this systematic offset as our sub-samples were also optically selected, although with very different criteria.

Our result from Fig. 16 that lower HI fractions could potentially lead to higher edge stellar surface densities can thus not be one due to a selection effect. It would certainly be worthwhile in future work to populate Fig. 16 with HI-rich ETGs (e.g. Serra et al. 2012) to confirm this result.

Appendix F: The use of scale lengths, fixed surface brightness, or isodensity radii measures as galaxy size indicators

Core to this paper is an attempt to overcome the limitations of past size indicators and locate a common ‘edge’ feature in galaxies of a large range in stellar mass and morphoplogy in the Fornax Cluster and other environments, connecting size to the outermost limits of star formation in a galaxy (see Trujillo et al. 2020; Chamba 2020; Chamba et al. 2022). As a reference for the reader, the median effective radii derived here for Fornax dwarfs are at 0.6±0.2 R_edge. The scale lengths derived by Su et al. (2021) via decomposition modelling of the disk (see their equations 5 and 6) of the same sample are also located close to this value. Therefore, it is already clear that the edge radii found here are inherently different from these past measures.

Isophotal radius measures have also been frequently used in the literature. For example, in the Koopmann et al. (2001) result mentioned earlier on the truncation of H_α radial profiles in Virgo galaxies, it is interesting to point out that these authors normalised their sample using an isophotal radius in the R-band at 24 mag/arcsec² (r₂₄). They also explored the use of disk scale lengths derived by first creating a r^1/4 bulge + exponential disk decomposition model and then locating the radius where the flux with respect to the centre has fallen by 1/e. However, they caution the reader about the difficultly in fitting such models to galaxies:

‘Given the systematic effects of differing surface brightnesses on isophotal radii [referring to the intrinsic variations between galaxies and its general sensitivity to background subtraction] and the difficulties of model fitting and unknown extinction values on disk scale length, it is not obvious which radial normalizer is a better indicator of galaxy/disk size.’ – Koopmann et al. (2001)

In other words, both fixed isophotal radii and scale lengths have their own limitations when considered as galaxy size indicators. While they found that their main conclusions are independent of which radii measure was used, their justification for their selection of r₂₄ was because the radius can be measured for more galaxies and that determining the scale lengths via modelling or fitting contributes a higher uncertainty; in other words, it was a pragmatic choice given the goals of their work.

We refer the interested reader to Trujillo et al. (2020) for an in-depth comparison between different size definitions, including the half-mass and Holmberg (1958) isophotal radii. Here we report that the median distance of the isophote at 24 mag/arcsec² in the r-band (close to r₂₄) is 0.4±0.2 R_edge and at 26 mag/arcsec² in the g-band is 0.7±0.2 R_edge for the Fornax dwarfs studied. In future work, it would be interesting to study the H_α radial profiles of Fornax galaxies for comparison with the optical edges reported here and the analysis performed in Koopmann et al. (2001).

Watkins et al. (2023) has also recently used the fixed iso-density contour at 3.63 M_⊙/pc² surface stellar density as a radius measure for FDS dwarfs. These authors find that the radius - mass relation significantly curves/drops towards smaller radii at lower masses M_⋆ < 10⁷ M_⊙ (see their Fig. 2). The outliers in this relation are those dwarfs with very low central densities. The authors argue that the outliers are low surface brightness galaxies, which are more susceptible to tidal forces, as they are found in the denser regions of the cluster. However, we point out that the visual morphologies of these outliers have not been reported or analysed in that work.

In contrast, we do not find the same drop in the edge radii-mass relation but a constant slope of 0.4 (Fig. 8). As addressed throughout Watkins et al. (2023), the density value of 3.63 M_⊙/pc² reaches the central regions of several dwarfs in their sample, corresponding to smaller or zero radii (see their Fig. 2). This fact implies that the contour is no longer representative of the outskirts of these galaxies or any structures at larger radii. For the sample of FDS dwarfs studied here, the 3.63 M_⊙/pc² iso-density radius when non-zero occurs at a median distance of 0.5±0.2 R_edge. This value is even slightly less than that achieved with the effective radii or scale lengths as discussed earlier, reflecting that the measure is more representative of the central or inner regions of these galaxies.

Similar to radius measurements dependent on light concentration, any fixed threshold will thus not reflect the same regions of galaxies across stellar mass as the variation of their inner structure is broad. This fact makes it even more difficult to decipher the environmental influence on scaling relations based on such measurements. However, if the reader is specifically interested in selecting the lowest density or morphologically diffuse dwarfs, such relations could perhaps present to be a useful guide.

All Tables

All Figures

Fig. 1. Visualisation of the problem with using effective (or half-light) radii as a size definition. A massive Milky Way-like galaxy (left) and an iconic low surface brightness, low mass dwarf galaxy (DF44; van Dokkum et al. 2016) (right), frequently thought to be as extended as the Milky Way, are shown at the same physical scale using images of similar sensitivity. In spite of the approximately four-orders-of-magnitude mass disparity, both of these galaxies have a comparable half-light radius (re; colored contours), partly explaining the large scatter in the re−mass relation and demonstrating that conventional measures of light distribution can be deceptive characterisations of total galaxy extension (Chamba et al. 2020). Thus, re may not be ideal to trace the environmental impact on size and its dependance on light concentration explains the lack of consensus in the literature.

In the text

Fig. 2. Radial profiles of galaxies with stellar masses ∼109 M⊙ from the Chamba et al. (2022) field sample. Left to right: RGB composite image from IAC Stripe 82 with a grey background to highlight the low surface brightness edge location (pink ellipse). Surface brightness profiles in r-band (orange) and g-band (cyan), g − r colour and surface stellar mass density profiles. The location of the detected edges are shown as vertical dashed lines and the shaded grey areas are the typical uncertainties associated with repeated measurements by Chamba et al. (2022) (∼0.04 dex).

In the text

Fig. 3. Schematic of the Fornax Cluster and Fornax A sub-group from the Fornax Deep Survey. The positions of dwarfs (purple stars), late-type and early-type galaxies (blue crosses and red dots, respectively) catalogued by Venhola et al. (2018) are plotted. The virial radius of the cluster (black circle with radius ∼2 deg) and Fornax A sub-group (black, dashed circle with radius ∼1 deg). The late-type galaxies in the subgroup are plotted as blue pluses for distinction.

In the text

Fig. 4. Distribution of the number of galaxies in stellar mass bins of width 0.5 dex in our final Fornax cluster (purple), satellite (yellow, right hatched), and nearly isolated (blue, dotted and left hatched) samples. The field sample combining the satellites and isolated galaxies are plotted in dashed black lines. C22* in this figure indicates that we have included the Chamba et al. (2022) LTG and ETG sample here only to show the lack of cluster galaxies beyond M⋆ > 109.5 M⊙ (grey region).

In the text

	Fig. 5. Distribution of the sample listed in Table 1 in stellar mass as a function of distance. The points are coloured according to their morphology, broadly categorised here as early- (red) or late-type (blue) [see text for details]. The markers in the legend label each sub-sample. The number of galaxies in each type are listed in Table 2.
In the text

Fig. 6. Similar to Fig. 2: example radial profiles of cluster dwarf galaxies with M⋆ ≲ 108 M⊙. The first panel is the gri-colour composite image using FDS images (Venhola et al. 2018), overlaid on a grey scale background to highlight the low surface brightness boundaries of these galaxies. White in this scale indicates non-detections or background pixels in the data. FDS names for each galaxy from Venhola et al. (2018), Su et al. (2021), the estimated stellar mass in units of log M⋆/M⊙ (left) and edge locations (dashed ellipses) are indicated in each panel. The second, third and fourth panels are the g (cyan) and r-band (orange) surface brightness, g − r and Σ⋆ profiles respectively. The identified edge feature is indicated as a vertical dashed line. To highlight the inflection point we mark as the edge feature in the colour and stellar surface density profiles, we additional plot a least squares polynomial fit of degree four as a solid line in those panels. Galaxies are ordered by increasing stellar mass from the upper to lower panels.

In the text

	Fig. 7. Similar to Fig. 6 but for satellite galaxies from ELVES Carlsten et al. (2021) within DECaLs (Dey et al. 2019).
In the text

Fig. 8. Size–stellar mass relation using edge radii (left) and the associated edge density–stellar mass relation (right). For visualisation reasons, we only plot the combined uncertainties due to the background and stellar mass estimation in our measurements (see text for details) for the field and Fornax LTGs/ETGs samples in the upper panels to demonstrate the typical values across the full stellar mass range of the scaling relations. Upper: Best-fit laws fitting the labelled dwarf samples: $R_{\mathrm{edge}}\sim M_{\star }^{1/3}$ (reported for galaxies with M⋆ > 107 M⊙ in Trujillo et al. 2020; Chamba et al. 2022) and $R_{\mathrm{edge}}\sim M_{\star }^{0.42}$ (this work). We plot the Fornax cluster dwarfs (purple pentagons), SAGA (Mao et al. 2021) and ELVES (Carlsten et al. 2022) satellites (yellow stars) and sub-sample Chamba et al. (2022) and Karachentsev et al. (2013) field galaxies (blue diamonds). For completeness, the radii for Fornax cluster massive late-type and early-type galaxies (black circles) are also shown. The shaded regions show the intrinsic scatter of each relation (∼0.06 dex for edge radii). At a fixed stellar mass, the majority of Fornax galaxies are below the best-fit line. The edge density-stellar mass plane is plotted using the same symbols and colours as in the left panels. The piecewise, Σ⋆, edge best-fit relations as well as the luminous ETGs and LTGs from Chamba et al. (2022) (grey crosses) are shown for reference. Middle: The same relations are now colour coded according to the global g − r colour of the galaxy. The symbols representing each sub-sample are those used in the upper panels. Lower: Similar to the middle panels, but now using the g − r colour to estimate the age of the galaxy from MILES (Vazdekis et al. 2012) for a fixed metallicity of [M/H] = –0.71 (see text for details). For reference, the age of the dwarf elliptical galaxies with M⋆ > 107 M⊙ in the Fornax Cluster is ∼10 Gyr (Rakos et al. 2001). The figure shows that galaxy edges in the Fornax cluster are smaller, denser and redder compared to the satellite and field samples. We use the best-fit relations to quantify and represent these results as histograms in Fig. 9.

In the text

Fig. 9. Distribution of edge radii (left) and surface stellar mass densities (right) normalised by the expected value derived from the regression of these measurements against stellar mass ($R_{\mathrm{fit}}\sim M_{\star }^{0.4}$ and ⟨Σ⋆, edge⟩ from Chamba et al. (2022), respectively; see Fig. 8). In the upper and middle panels, Fornax dwarf galaxies are shown in purple. The individual samples considered are separated by the labelled designs (upper). The distribution of the rest of the sample (field + satellites combined) is plotted in black dotted lines. In the middle panel, we combine the field (Chamba et al. 2022 and Karachentsev et al. 2013 in blue) and satellite (SAGA; Mao et al. 2021 and ELVES; Carlsten et al. 2022 in yellow) samples to further visualise the distinction between their distributions. The lower panels show the distribution of galaxies in the size– and edge density–stellar mass planes when separated in three bins of g − r colour, showing that redder, older galaxies populate the lower half of the size–mass relation (left) and upper half in the density-stellar mass relation (right). Quantitatively, the panels show that compared to field galaxies, Fornax cluster galaxies can be up to 50% smaller and on average, redder, older and two times denser at the edge.

In the text

	Fig. 10. Scaling relations colour-coded by morphology. Early-type galaxies are coloured in red are smaller on average than the late-type galaxies in blue across the stellar mass range studied. The dotted and dashed lines are the same as those previously plotted in the upper panels of Fig. 8.
In the text

	Fig. 11. Effective radius–stellar mass plane for galaxies in different environments. In the top panel, the data points are labelled as in Fig. 8. The lower panel is similar to Fig. 10. We colour-coded the effective radius–stellar mass relation, using red for the early-type and blue for the late-type galaxies. In both panels, the shaded regions correspond to the 0.2 dex scatter for reference.
In the text

Fig. 12. Distribution of half-light radii, normalised by the expected value derived from the regression of re against stellar mass ($r_{\mathrm{fit}}\propto M_{\star }^{0.28}$; see Fig. 11). As in Fig. 9, Fornax dwarf galaxies are shown in purple. The upper panels shows the distribution of individual samples separated by the labelled designs as in the left of Fig. 9. The lower panel shows the full satellites (yellow) and the combined nearly isolated + satellites samples (blue). The total field sample is plotted in dotted black lines. There is no obvious distinction between the samples in the effective radius–stellar mass plane.

In the text

Fig. 13. Median effective (upper) and edge radii (lower) in bins of stellar mass. The distribution of the number of galaxies used to compute the median radii in each stellar mass bin was plotted in Fig. 4. The shaded grey region corresponds to the stellar mass regime beyond which we lack cluster and satellite galaxies for a meaningful, statistical comparison. However, we compute the median points with the few galaxies in our sample for completeness. The separation between the Fornax cluster and field sample is much more clear in median edge radii: the cluster galaxies are systematically smaller in edge radii across the full stellar mass range considered, which is not reflected by the re measurements.

In the text

Fig. 14. Similar to Fig. 13: median radii for early- and late-type galaxies in our total sample. The error bars correspond to three times the dispersion in the estimated median value. Early-type galaxies are larger in effective radius when M⋆ ≲ 108 M⊙ compared to the late-type galaxies (upper). However, in their edge radii (lower), the early-type galaxies are systematically smaller by ∼20% than the late-type ones across the full stellar mass range.

In the text

Fig. 15. Similar to Fig. 13 but now separated by morphology and environment. The early-type galaxies are plotted on the left and late-type ones on the right. In both panels we plot those galaxies in the field by combining the nearly isolated and satellite samples (green squares). We also plot the combined line for the cluster using the full sample of early- and late-type galaxies (the same shown in Fig. 13) in the dot-dashed line. Both the early- and late-type galaxies are smaller in the Fornax cluster compared to the field. However, the early-type galaxies are even smaller than the late-type ones, regardless of the environment (Fig. 14).

In the text

Fig. 16. Ratio of HI-to-stellar masses of a sub-sample of local volume field galaxies (blue diamonds; 64 studied here out of the 96 with available DeCaLs imaging from Karachentsev et al. 2013); nine SAGA and 15 ELVES satellites (yellow stars; matched with the Durbala et al. (2020) and Zhu & Putman (2023) measurements, respectively); and the Fornax galaxies (purple pentagons; 17 studied here out of the 39 HI masses or upper limits reported in Kleiner et al. 2023) as a function of edge surface stellar mass density. The HI masses of the Fornax LTGs and ETGs are taken from Loni et al. (2021) when available (dark blue points). Higher edge densities are found when the H I-to-stellar masses ratios are lower. A best-fit to the data points is shown as a dashed line to highlight this trend.

In the text

	Fig. A.1. Radial profiles of the dwarf galaxy FCC035 using different bin sizes. The bin sizes are in pixel units. While increasing the bin size reduces the noise in the profiles as expected, the shape of the profiles and the feature we identify as the location of the edge do not change.
In the text

Fig. B.1. Radial profiles of asymmetric galaxies from the Raj et al. (2019) LTG sample. These galaxies are within the virial radius of the main Fornax cluster and are clearly asymmetric in some form (see text for details). In all cases, our default radial profile derivation permits the identification of an edge feature in the colour or density profiles (vertical dashed line). However, as we are using fixed elliptical annuli, our profiles should be considered as an average representation of the data.

In the text

	Fig. C.1. Reproducing the scaling relations and histograms in Fig. 11, 12, and 13 using the modelled effective radii values from Su et al. (2021) for Fornax galaxies. Our main conclusion that the effective radii of Fornax galaxies are not significantly different from those found in the field for low stellar mass galaxies < 109 M⊙ remains unchanged.
In the text

Fig. D.1. Same size–stellar mass relations as in Fig. 8 but colour-coded according to their age as estimated using the g − r colour at the fixed metallicity value stated in the colour bar. While the differences between the relations using -1.71 and -1.31 are minor, there are larger differences in individual points with the -0.71 relation. However, given that we are interested in relative age distributions, the bifurcation we report in this work remains unchanged when considering either of those relations separately (see text for discussion).

In the text

Fig. E.1. HI-stellar mass relation (upper) and the HI-stellar mass ratio as a function of stellar mass (lower) for a sub-sample of Fornax, satellites, and field galaxies in our work as labelled. Arecibo Observatory and Green Bank Telescope targeted observations by Geha et al. (2006) of optically selected dwarfs and ALFALFA detections (both for SDSS galaxies) from Durbala et al. (2020) are also included for reference. The offsets between the datapoints in our sub-samples, the Geha et al. (2006) work and ALFALFA is expected when galaxies are optically selected (see text for details). Arrows show HI upper limits where reported in the respective catalogues.

In the text