© ESO 2020

1. Introduction

The stellar mass (M_*) and gas-phase metallicity (Z) are among the most fundamental properties of galaxies and tracers of galaxy formation and evolution. While the stellar mass provides information about the amount of gas that is locked up into stars, the metallicity gives indications on the history of star formation and the exchange of gas between an object and its environment.

The mass-metallicity relation (MZR) has been studied for several decades, beginning with the pioneering work of Lequeux et al. (1979). Since then, several authors have investigated the origin and behaviour of this relationship in low and high redshifts, and in different environments and mass regimes. Tremonti et al. (2004) studied this relation for > 53 000 star-forming galaxies (SFGs) in a demonstration of the statistical power of the Sloan Digital Sky Survey (SDSS). They found a tight (±0.1 dex) correlation between stellar mass and metallicity spanning over three orders of magnitude in stellar mass (down to 10^8.5 M_⊙) and a factor of 10 in metallicity, with a median redshift of 0.1. Possible explanations of the origin of the MZR include (i) gas outflows. In this scenario, low-mass galaxies suffer a stronger effect because they have lower escape velocities and can more easily loose enriched gas through stellar winds than massive galaxies. This scenario was first proposed by Larson (1974, see also Spitoni et al. 2010; Tremonti et al. 2004). Furthermore, Belfiore et al. (2019) recently found that the inferred outflow loading factor decreases with stellar mass. (ii) The second possible explanation might be the downsizing effect: massive galaxies process their gas faster, on shorter timescales and at earlier epochs than low-mass galaxies, where the star formation is slower and extends over longer periods (e.g. Cowie et al. 1996; Thomas et al. 2010). (iii) Another possible mechanism for the MZR is the dependence of stellar mass or star formation rate (SFR) on the initial mass function (IMF; e.g. Köppen et al. 2007; Gunawardhana et al. 2011). However, it is still a matter of debate which of these mechanisms or which combination of them plays a more important role in the formation and evolution of galaxies.

With the advent of large surveys, there has been an effort to observe fainter and higher redshift objects because they would provide strong constraints on our understanding of how galaxies evolve. For instance, the Galaxy and Mass Assembly (GAMA; Driver et al. 2011; Liske et al. 2015) survey observed ∼300 000 galaxies two orders of magnitude deeper than SDSS, covering an area of ∼286 deg². In the survey, an evolution of metallicity of ∼0.1 dex was found for massive galaxies at redshift ∼0.35 (Lara-López et al. 2013). At higher redshifts, the VIMOS Very Large Telescope (VLT) Deep Survey (VVDS; Garilli et al. 2008) provides spectroscopic data of > 35 000 galaxies up to redshift z ∼ 6.7 over a total of > 8 deg² down to i_AB = 24.75. Using VVDS data, Lamareille et al. (2009) studied the MZR up to z ∼ 0.9. They defined two samples: a wide (i_AB < 22.5) and a deep sample (i_AB <  24). They used three different metallicity calibrators depending on the redshift range. All methods were then normalized to the Charlot & Longhetti (2001) calibrator, which is in agreement with the Tremonti et al. (2004) metallicities. Assuming that the MZR shape is constant with redshift, they found a stronger metallicity evolution in the wide sample and concluded that the MZR is flatter at higher redshift. They arrived at the same conclusion for the luminosity-metallicity relation (LZR) in the wide sample.

There is an incentive to also analyse the low-mass end of the MZR because a flattening in this relation has been reported. For instance, Jimmy et al. (2015) found a flattening of the MZR relation using a sample of local (D < 20 Mpc) dwarf galaxies with the VIMOS integral field unit (IFU) spectrograph on the VLT. They also found a clear dependence of the MZR and LZR on SFR and H I-gas mass. They used this finding to explain the observed scatter at the low-mass (< 10⁸ M_⊙) end of these relations. Using a sample of 25 dwarf galaxies, Lee et al. (2006) also extended the MZR for galaxies down to a M_* ∼ 10⁶ M_⊙, roughly 2.5 dex lower in stellar masses than Tremonti et al. (2004). Using Spitzer mid-IR (MIR) (4.5 μm) photometry, they found a comparable scatter in the MZR over 5.5 dex in stellar mass. They concluded that if galactic winds are responsible for the MZR, increasing dispersion is expected, as they observed in their analysis.

A clear extreme in the MZR is given by extremely metal-deficient (XMD) galaxies (≤0.1 Z_⊙). As found by Thuan et al. (2016), XMD galaxies show a clear trend of increasing gas mass fraction with decreasing metallicity, mass, and luminosity. On the same line, Sánchez Almeida et al. (2014, 2015) have argued that the low metallicities of the XMD galaxies are an indicator of infall of pristine gas. However, Thuan & Izotov (2005) found that low metallicity alone cannot explain the hard ionising radiation observed in blue compact dwarf (BCD) galaxies, and proposed fast radiative shocks (velocity > 450 km s⁻¹) associated with supernovae explosions of massive Population III stars. Ekta & Chengalur (2010) examined the MZR of XMD galaxies and compared it with the MZR of BCD and dwarf irregular galaxies. They proposed that XMD galaxies are extremely metal depleted due to better mixing of the inter-stellar medium (ISM).

The fact that there are outliers of the MZR makes the situation even more interesting. Peeples et al. (2008) found a sample of 41 local (z <  0.048), low-mass, high-oxygen abundance outliers from the Tremonti et al. (2004) SDSS-based relation. They argued that these compact, isolated, and morphological undisturbed sources may be transitional dwarf galaxies. Their red colours (approaching or on the red sequence) and low gas fraction suggest that they are in the final stage of their star formation period. Moreover, using a larger sample of galaxies from the SDSS and DEEP2 surveys with stellar masses down to 10⁷ M_⊙, Zahid et al. (2012) found that the properties of low-mass metal-rich galaxies (the outliers from MZR) are, in agreement with previous works, possible transitional objects between gas-rich dwarf irregular and gas-poor dwarf spheroidal and elliptical galaxies.

In consideration of previous studies, we wish to shed light on the low-mass end of the MZR. To this aim, we used data from the OSIRIS Tunable filter Emission-Line Object (OTELO) survey reported by Bongiovanni et al. (2019, hereafter OTELO-I) to study a sample of low-mass Hα emission line sources (ELS) at redshift ∼0.4, firstly described in Ramón-Pérez et al. (2019, hereafter OTELO-II). As demonstrated in Bongiovanni et al. (2020), OTELO has allowed validation of the narrow-band (NB) scan technique using tunable filters (TFs) for finding faint populations of SFGs, which makes this survey the most sensitive to date in terms of minimum emission-line flux (∼5 × 10⁻¹⁹ erg s⁻¹ cm⁻²) and observed equivalent width (EW ≈ 6 Å) achieved.

This article is organized as follows. In Sect. 2 we present the data and sample selection process together with its characteristics and the external catalogues we used. In Sect. 3 we describe the methods for estimating stellar mass, gas metallicity, SFR, and morphology determination from OTELO data for the science case of this study. In Sect. 4 we present our analysis and discussion, followed with our conclusions in Sect. 5. When necessary, we adopt the cosmology from Planck Collaboration XIII (2016) with Ω_Λ = 0.69, Ω_m = 0.31, and H₀ = 67.8 km s⁻¹ Mpc⁻¹.

2. Data and sample selection

We study a sample of Hα ELSs with stellar masses in the range of 10^6.8 <  M_*/M_⊙ <  10¹⁰, which places our sample in the low-mass regime. To this aim, we used data products of OTELO. A brief description follows.

The core of the OTELO survey consists of red tunable filter (RTF) observations, which were carried out in a total of 108 dark hours with the OSIRIS (Cepa et al. 2000) instrument at the 10.4 m Gran Telescopio Canarias¹ (GTC). These observations cover a ∼56 arcmin² section of the Extended Groth Strip (EGS) region in six dithered pointings. The nominal central wavelengths at the centre of the field of view (FOV) range from 9070 to 9280 Å with Δλ of 6 Å steps. A full width at half-maximum (FWHM) of 12 Å was adopted for each one of the resulting 36 RTF tunings. This sampling of the RTF is optimal for deblending the Hα and [N II]λ6583 lines (Lara-López et al. 2010a) for systems at z ∼ 0.4. In addition, all RTF images were combined in a single OTELO_deep image. A source detection on this image was performed using the SExtractor software (Bertin & Arnouts 1996), providing 11 237 sources. The integrated flux (OTELO_Int) of the sources detected in this image has a limiting magnitude of 26.4 [AB] (50% completeness).

The OTELO_deep image was also used to measure the point spread function (psf)-model fluxes of these 11 237 sources on registered and resampled public images in the optical and near-infrared (i.e. u, g, r, i, z, J, H and Ks bands) obtained from the AEGIS collaboration². These data were cross-matched with public catalogues in X-rays, far-UV (FUV) and near-UV (NUV)³, as well as those from Herschel and Spitzer IR bands to obtain additional photometric data for the sources so correlated. In addition, high-resolution F606W and F814W image stamps from the ACS of the Hubble Space Telescope (HST) of each source, when found, were obtained for further studies, in particular for morphological classification.

Related to the scope of this work, the resulting spectral energy distributions (SED) for each OTELO source were processed with the LePhare⁴ code (Arnouts et al. 1999; Ilbert et al. 2006) using a galaxy template library with the four standard Hubble types E/S0, Sbc, Scd, and Irr from Coleman et al. (1980) and six SFGs templates from Kinney et al. (1996), and an active galactic nucleus/quasi-stellar object (AGN/QSOs) library with two Seyfert, three type-1 and two type-2 QSOs and three composite (starburst+QSO) templates created by Polletta et al. (2007). This procedure allowed us to obtain photometric redshift solutions z_phot, which were independently calculated including and excluding the OTELO_deep photometry in the input SED. This provided a set of possible z_phot solutions, along with the two best-fit SED templates and the corresponding extinction E(B − V) estimates. The final best solution was obtained by the analysis of the best χ² solution obtained together with the estimated redshift error, σ_Zphot, defined by

$$\begin{array}{c}\hfill {\sigma }_{\mathrm{Zphot}}=|\mathtt{Z}\_\mathtt{BEST}\_\mathtt{LOW}\_\mathtt{deepX}-\mathtt{Z}\_\mathtt{BEST}\_\mathtt{HIGH}\_\mathtt{deepX}|,\end{array}$$(1)

where Z_BEST_LOW_deepX and Z_BEST_HIGH_deepX are the 1σ deviations of the estimated redshift for the given object, while X stands for Y or N, including or excluding the OTELO_deep photometry in the SED fitting, respectively. Finally, the astrometric and photometric data for each source, along with the photometric redshift estimates and the redshift data from other surveys DEEP2 (Newman et al. 2013) and CFHTLS T0004 Deep3 (Coupon et al. 2009), were included in the dubbed OTELO multi-wavelength catalogue.

The RTF scan data were also used to provide a pseudo-spectrum (PS) for each entry in the OTELO multi-wavelength catalogue. The PS are properly calibrated in flux and wavelength and differ from conventional spectra in that the former are affected by the distinctive RTF transmission profile. Such PS typically contain 36 wavelength points (R ∼ 700), but the actual number of points and the wavelength range covered by each PS depend on the position of the object in the FOV. Some PS examples are given in Fig. 1. For each PS a pseudo-continuum flux (f_c), which itself is defined as the median flux of the PS, and a deviation around f_c (σ_med,  c) were estimated. An object was qualified as an ELS candidate when either two adjacent points of the PS were above f_c + 2σ_med,  c, or only one was above it but it has an adjacent point above f_c + σ_med,  c. With these criteria, a list 5322 preliminary ELS candidates for the whole survey was obtained. Further details about the survey design, observations, data reduction, photometric redshift estimations, the construction of the multi-wavelength catalogue, the PS preparation, and the preliminary ELS selection are given in OTELO-I.

Fig. 1. Pseudo-spectra of selected sources. From left: source id: 2146 with a clear Hα + [N II] emission line complex at z = 0.387; source id: 625 with Hα (truncated in the blue side) and [N II] emission lines and [S II]λ6716 (on the red side of the spectrum) at z = 0.371; and source id: 3289, which exhibits the [S II]λλ6716,6731 doublet at z = 0.340. The green line shows the best-fit synthetic spectrum after the convolution process. See Sect. 3.2 for details.

2.1. Hα + [N _II] sample selection

We used the OTELO data described above to select the Hα sample as follows:

(a) According to OTELO-II, we selected the sources where the best z_phot (including and excluding the OTELO_deep photometry) was in the range 0.2 <  z_phot <  0.5 with the further constraint that σ_zphot <  0.2 × (1 + z_phot). The total number of sources in this redshift window is 2289. In this selection we lost some Hα + [N II] candidates whose best z_phot would be obtained with AGN/QSOs libraries instead of galaxy templates. This does not affect the goals of this work. In addition, we are also aware that our selection includes possible [S II]λλ6716,6731 at z ∼ 0.3. The best-fit templates correspond to Scd, Im, or SFGs types, with the exception of two sources where the best z_phot was provided by an E/S0 galaxy type. Within this set of 2289 sources, we found 73 preliminary ELS candidates to Hα + [N II].

(b) We visually inspected the 73 candidates using our web-based graphic user interface (GUI)⁵ where all data about a user-selected source can be displayed; in particular, it includes the PS, the thumbnails of the images in available bands, and the SExtractor segmentation maps together with high-resolution HST images, z_phot solution including and excluding OTELO_deep photometry obtained from our photo-z analysis and their comparison with the input photometry, the z_phot solution from the CFHTLS T0004 Deep3 photo-z catalogue, and the z_spec from DEEP2.

The high-resolution HST-ACS images allow us to identify possible contamination of the PS from neighbouring sources that are not resolved in the ground-based images. The comparison of the best-fit SED and the source data, together with the additional redshift estimates (obtained in our analysis, from the CFHTLS z_phot catalogue, and from the objects with DEEP2 z_spec) allows us to evaluate the confidence in our z_phot solution. Finally, the PS allows us to distinguish between Hα + [N II] SFG sources, [S II]λλ6716,6731, or AGN galaxies with Hα + [N II] emission. We took advantage of the fact that the [N II]λ6583 over Hα ratio must be lower than 0.4 (using the extreme case used by Pettini & Pagel 2004, see below) in SFGs. In addition, the [S II]λ6716 over [S II]λ6731 line ratio must range between 1.49 and 0.44 (see McCall 1984; Cedrés et al. 2013 in the context of tunable filters). This means that any doublet where the redward line ([N II]λ6583 or [S II]λ6731) is stronger than about half of the blueward line (Hα or [S II]λ6716) should be a Hα + [N II] AGN or a [S II]λλ6716,6731 source. This process reduced the sample of 73 candidates to a sample with 18 sources where Hα and [N II]λ6583 can be measured. In addition, we also obtained a z_GUESS value (from the web-based GUI), which was obtained by the association of the brightest RTF slice and the Hα emission line at a given observed wavelength.

(c) As a final cross-check, we searched for objects with DEEP2 spectroscopic redshift in the range of 0.35 ≤ z_spec ≤ 0.42, which covers the range where Hα line would be detected in the OTELO spectral window. This redshift range would overlap with [S II]λλ6716,6731 sources (0.32 ≤ z ≤ 0.38). DEEP2 data provides 18 sources in common with our selection in point (b), 9 of them do not show any emission feature in the OTELO PS, and 7 of them are in common with our selection from point (b). The remaining 2 objects had a best z_phot solution for the galaxy library outside of our redshift selection range (although one of them provides a z_phot solution in the Hα + [N II] range using the AGN/QSOs library with a Sy-2 galaxy template). After visual inspection of the PS, we verified that one source of this set is indeed a Hα + [N II] emitter (id: 625) and the second one displays only the [N II]λ6583 line (and [S II]λλ6716,6731 lines on the red side, id: 6059) in its PS. We therefore included only id: 625 source in the final sample, which increases to 19 objects. We note that this additional source was not included in the preliminary ELS list because the emission line is truncated in one extremity of the PS (see the middle panel of Fig. 1), but it was selected as a Hα + [N II] SFG candidate with the colour-magnitude ELS selection technique that is traditionally used in deep NB surveys (Pascual et al. 2007). In our case, a (z-OTELO_Int) colour was obtained for all the OTELO sources detected in z band. See OTELO-I for details about this ELS selection procedure.

The recovery of a preliminary ELS through a colour-magnitude diagram instead of using the flux-excess in PS, as indicated above, is an exception. As can be inferred from the description of this survey, OTELO combines the advantages of the deep NB surveys that imperatively use the colour-magnitude selection technique, but whose sensitivity to low values of observed EW is constrained by the passband of the NB filter used, with the strengths of the conventional spectroscopic surveys but without their selection biases and long observing times required. This has allowed us to reach the line flux and EW sensitivities reported in Sect. 1.

In summary, the final sample of Hα ELSs SFG candidates consists of 19 sources; we refer to this hereafter as the Hα sample. Further refinements (i.e. the flux and metallicity estimations) are described in the following sections. The examples given in Fig. 1 show a PS for a Hα + [N II] source (id: 2146), a source with a truncated Hα (id: 625), and [S II]λ6716 emission and a [S II]λλ6716,6731 source (id: 3289, not in the Hα sample). The top part of Table 1 shows a summary of the number of sources at different steps of the selection process. In the appendix we show figures with the PS of the sources that compose the Hα sample.

2.2. Data from previous surveys

In order to compare our results with those from previous studies, we used the SDSS-Data Release 7 (SDSS-DR7, Abazajian et al. 2009) and the GAMA survey (Driver et al. 2011). From both datasets we considered only the SFGs selected through the BPT diagram (Baldwin et al. 1981) using the criteria of Kauffmann et al. (2003). In both surveys, the stellar masses were estimated using SED templates from Bruzual & Charlot (2003) and a Chabrier (2003) IMF. SDSS-DR7 fit templates to the observed SEDs, while stellar masses in GAMA were estimated using (g − i) colours and a mass–luminosity relation (Taylor et al. 2011).

We selected the following sub-samples from the SDSS-DR7 (local and high-z) and the GAMA surveys: (i) A local SDSS sample with 0.005 <  z <  0.1 and a signal-to-noise ratio (S/N) > 3 for Hα and S/N >  4 for the [N II] emission line. (ii) A higher redshift sample (high-z SDSS and GAMA) with 0.3 <  z <  0.33 (upper-z limit due to the Hα line visibility in both surveys). The emission line S/N imposed for both high-z samples was the same as in (i).

Furthermore, to ensure completeness for the higher redshift samples, we imposed a cut in stellar mass of 10¹⁰ M_⊙. This cut is given by the limit in completeness of SDSS (see Weigel et al. 2016), and it approximately corresponds to an I-band absolute magnitude limit of M_i ∼ −21 [AB]. These selection processes guarantee that we used robust and complete samples from both surveys.

We also used the data from publicly available catalogues of the VVDS⁶, in particular, from VVDS-Deep and VVDS-Ultra-Deep (VVDS-F0226-04). Data used from both surveys were selected as follows: redshift quality flag 1 < ZFLAG < 11 (this removes possible AGN from the sample, as described in Le Fèvre et al. 2013); the redshift range of 0.3(Deep)/0.2 (Ultra-Deep) < z < 0.42, we decided to use z = 0.2 as the lower redshift limit for Ultra-Deep in order to increase the final number of sources.

3. Methods

3.1. Stellar masses

Stellar masses M_* for the overall OTELO sample were computed using the López-Sanjuan et al. (2019; hereafter LS18) mass-to-light ratio for quiescent and SFGs (their Eqs. (10) and (11), respectively) employing rest-frame g- and i-band magnitudes, together with absolute i-band magnitude M_i. The rest-frame photometry was computed by shifting the SED_best (see Sect. 2) to z = 0, and calculating the flux below the transmission curves for g, r, i, z (CFHTLS) and for the HST-ACS F606W and F814W filters. To estimate M_i we used our z_OTELO (see Sect. 3.2). A second-order correction accounted for the Galactic dust extinction towards the EGS using the Galactic dust map from Schlafly & Finkbeiner (2011). The obtained stellar masses M_* for the Hα sample are in the range of ∼10^6.8 <  M_* <  10¹⁰ M_⊙. This places our sample in a very low-mass regime where 63% of our sample has M_* <  10⁹ M_⊙. To test its consistency, we also calculated M_* using the Taylor et al. (2011; hereafter T11) mass-to-light ratio. Both methods agree, with the most noticeable difference for blue SFGs, for which LS18 provides an additional quadratic factor that depends on the (g − i) colour. In Fig. 2 we compare the computed stellar masses using prescriptions from LS18 and T11 for all the OTELO sources (median value of M_* = 10^8.9 M_⊙) and for the Hα sample (median value of M_* = 10^8.4 M_⊙). In Fig. 3 we show the obtained stellar masses as a function of rest-frame (g − i) colour for the overall OTELO and the Hα ELS samples. For comparison, in the same figure, we show the SDSS overall sample together with the blue and red cloud empirical colour division of Bluck et al. (2014):

$$\begin{array}{c}\hfill (g-r)=0.06\times logM[M_{\odot }]-0.01,\end{array}$$(2)

Fig. 2. Comparison of computed stellar masses M* using prescriptions from López-Sanjuan et al. (2019) and Taylor et al. (2011). The overall OTELO and Hα sample are shown as black dots and black empty circles, respectively. Dashed-line in blue represent 1:1 line.

obtained using data from SDSS-DR7. After we identified blue and red clouds in SDSS using the (g − r) colour, we represented them in terms of (g − i). Because it is only for illustrative purposes, we did not correct the blue and red cloud selection in terms of colour used.

Furthermore, we compared the obtained masses with the COSMOS2015 catalogue (Laigle et al. 2016) finding good agreement: our mass and colour estimates fall within the COSMOS2015 imprint. For the sake of clarity, we decided to omit this in Fig. 3, where a similar comparison is shown using data from SDSS-DR7. The resulting stellar masses of the sample we used and their associated uncertainties are discussed in Appendix B and shown in the fourth column of Table B.1.

Fig. 3. Colour–M* diagram. Blue points show Hα ELS, and green contours represent the overall OTELO sample. Filled contours in grey show the envelope of SDSS-DR7 data, and filled red and blue contours show the red and blue cloud separated with the empirically estimated limit of Bluck et al. (2014). Green squares mark MZR outlier galaxies (see Sect. 4.1).

3.2. Deconvolution of PS, metallicity, SFR, and sSFR

To obtain gas-phase metallicities and AGNs contamination, we used the N2 index:

$$\begin{array}{c}\hfill \mathrm{N}2=log\frac{f([\text{N}\text{II}]\lambda 6583)}{f(\mathrm{H}\alpha ),}\end{array}$$(3)

which requires a flux estimate. To this end, we used the so-called inverse deconvolution of the PS. In short, we assumed rest-frame model spectra defined by a constant continuum level, and Gaussian profiles of the [N II]λ6548, Hα, and [N II]λ6583 lines defined by their amplitude and a common line width for the three lines. The model to be compared with observational data is thus defined by the parameters f_mod(z, f_c, σ_line, f_{[N II] λ6548}, f_Hα, f_{[N II] λ6583}). We performed 10⁶ Monte Carlo simulations for each object and varied the different parameters. The different parameters varied in following ranges. The redshift was varied in the range z_GUESS ± 0.001. The line width σ_line varied from 20 to 500 km s⁻¹ rest-frame values, which translates into a range of 0.43−11 Å in spectral units at the Hα wavelength. The continuum f_c level varied in the range f_med, c ± σ_med, c with f_med, c the median of the PS and σ_med, c the root mean square around f_med, c, as defined in Sect. 2.1. The rest-frame f_Hα amplitude varied between f_med, c and the maximum value observed in the PS, $f_{\mathrm{PS}}^{\max }$ taking into account the observational error $\sigma (f_{\mathrm{PS}}^{\max })$, and taking into account the assumed Gaussian profile. This is is given by Eq. (A.5). The rest-frame f_{[N II] λ6583} varied between 0.006 ⋅ f_Hα and the maximum possible value given by Eq. (A.5). Finally, the rest-frame f_{[N II] λ6548} was fixed to 1/3 ⋅ f_{[N II] λ6583} (Osterbrock & Ferland 2006). See Appendix A for details.

Each model was convolved with the OTELO filter response to produce a synthetic PS that was compared with the observed PS, and the corresponding χ² was obtained. The Monte Carlo simulations are designed to sample the multiparametric probability density function (PDF) from which modal values (i.e. best-fit solutions) and different confidence intervals around the modal value were obtained for each parameter (see details in Appendix A). In the following we consider error bars as the confidence intervals, which includes 25% and 68% (which would be the equivalent of ±1σ for the Gaussian case) of the total area of the PDF around the modal value as an estimate of the errors. The redshift value obtained for each source after deconvolution (hereafter z_OTELO) are given in Table B.1. Formally, the common maximum uncertainty of these values is about 0.001(1 + z_OTELO), as established in Bongiovanni et al. (2020).

We note that the possible flux-related parameters we obtained are strongly correlated, that is, each model has its own N2 index (i.e. metallicity), line equivalent widths, and so on, because the variation in one of the parameters of the model non-trivially affect the possible best solutions of the other parameters. In this way, we also obtained the corresponding PDFs of all the associated parameters (equivalent widths, N2 indices, and oxygen abundances). For all these inferred quantities we performed a posteriori operations over the entire Monte Carlo set associated with each particular object. The corresponding PDFs (and associated errors) of the particular quantities can be obtained as shown in Appendix A. In the case of f_Hα, we corrected the model values for possible stellar absorption of underlying older components following Hopkins et al. (2003, 2013) using a EW_c correction value of 2.5 Å:

$$\begin{array}{c}\hfill F_{\mathrm{H}\alpha }^{\mathrm{cor}}=\frac{{\mathrm{EW}}_{\mathrm{H}\alpha }+{\mathrm{EW}}_{\mathrm{c}}}{{\mathrm{EW}}_{\mathrm{H}\alpha }}\times F_{\mathrm{H}\alpha }^{\mathrm{mod}},\end{array}$$(4)

where $F_{\mathrm{H}\alpha }^{\mathrm{mod}}$ and $F_{\mathrm{H}\alpha }^{\mathrm{cor}}$ are the model before and after stellar absorption correction, respectively, and EW_Hα refers to the equivalent width of the model before correction.

Finally, we computed the gas-phase oxygen abundance from the N2, with the absorption corrected $F_{\mathrm{H}\alpha }^{\mathrm{cor}}$ value by means of Eq. (1) of Pettini & Pagel (2004) for all simulations:

$$\begin{array}{c}\hfill 12+log(\mathrm{O}/\mathrm{H})=8.90+0.57\times \mathrm{N}2.\end{array}$$(5)

Nominal values and errors for metallicities were calculated by analysing the metallicity PDF obtained from the Monte Carlo simulations, but we did not take the intrinsic dispersion in the calibration into account, which would be ±0.41 (±0.18) dex at 95% (68%) confidence interval (Pettini & Pagel 2004). Table B.1 shows the resulting f_Hα, EW_Hα (both including the correction for stellar absorption), f_{[N II] λ6583}, and 12 + log(O/H) (without calibration uncertainty) values with their 68% confidence intervals in Cols. 6, 8, 7, and 5, respectively.

We note that by construction, the inverse deconvolution process always provides a solution for the intensity of the f_{[N II] λ6583} value, which would correspond to a 12 + log(O/H) = 7.63 (before the stellar absorption correction). These values would be below the OTELO line flux limit of 5 × 10⁻¹⁹ erg⁻¹ s⁻¹ cm⁻² as occurs for five objects (see Bongiovanni et al. 2020, for the flux limit estimation). These cases translate into a large uncertainty in the resulting N2 and metallicity values. We refer to these objects as low-flux sources and can only provide a f_{[N II] λ6583} (hence metallicity) limit.

In addition, the Pettini & Pagel (2004) method is valid in the range of −2.5 < N2 < −0.3, which corresponds to the 7.47 to 8.73 12 + log(O/H) range. Values of 12 + log(O/H) equal to 8.67 would correspond to N2 = −0.4, which defines an empirical division between SFGs and AGNs (Stasińska et al. 2006). In order to remove possible AGNs from our sample, we used the same method as in Ramón-Pérez et al. (2019, as for narrow-line AGN), that is, the Cid Fernandes et al. (2010) diagnostic diagram (EW_Hα vs. N2) shown in Fig. 4. In Fig. 4, the green and blue dashed lines are the limits of the empirical metallicity estimations from Yin et al. (2007) and Pettini & Pagel (2004) of −0.5 and −0.3, respectively. In this way, we found three sources as possible AGN and excluded them from the further analysis. The lower part of Table 1 shows the summary of sources of different classes after the inverse deconvolution and AGN discrimination process.

Fig. 4. N2 index vs. EWHα. Blue points show all the Hα candidates. Black and golden circles represent sources with an [N II] line flux above and below OTELO flux limit, respectively (see Sect. 3.2 and Bongiovanni et al. 2020). Red circles show AGN candidates selected using the Stasińska et al. (2006) empirical limit of N2 = −0.4. Vertical green and blue dashed lines show the Yin et al. (2007) and Pettini & Pagel (2004) limits of their calibration using the N2 index, respectively. The red line shows the AGN selection limit from Stasińska et al. (2006). The error bars in each object correspond to the 25% and 68% confidence interval around the most probable value.

Finally, we corrected the Hα fluxes for dust attenuation using the reddening values obtained from LePhare SED fitting (quoted in the second column of Table B.1), and the empirical relation by Calzetti et al. (2000) estimated for galaxies at z = 0.5 by Ly et al. (2012). We note that this is a lower value of L(Hα) because the extinction is obtained from the fit of galaxy templates whose intrinsic extinction is unknown. The derived values of E(B − V) are in the range of 0.0 and 0.3 mag, which is consistent with the extinction expected for low-mass galaxies, as shown in previous studies (e.g. Domínguez Sánchez et al. 2014). We also note that LePhare does not provide an evaluation of the uncertainty in E(B − V), therefore the uncertainty in this term was not taken into account to evaluate the uncertainty in L(Hα) (hence in the SFR, see Appendix B) The Hα line fluxes were used to derive SFRs following the standard calibration of Kennicutt (1998), but including the correction for a Chabrier (2003) IMF (see Kennicutt & Evans 2012, their Table 1), which is the one used in this work:

$$\begin{array}{c}\hfill \mathrm{SFR}({\mathrm{M}}_{\odot }{\mathrm{yr}}^{-1})=5.37\times {10}^{-42}L(\mathrm{H}\alpha )[\mathrm{erg}{\mathrm{s}}^{-1}].\end{array}$$(6)

The specific SFR (sSFR) was subsequently estimated as SFR/M_*. The resulting SFR and its associated uncertainty are listed in Col. 9 in Table B.1.

3.3. Morphology

Because the galaxies we analysed are some orders of magnitudes fainter than those in other surveys, it is mandatory to characterize this population as well as possible. Therefore, we performed a (i) quantitative and (ii) qualitative morphological classification. In both cases we decided to base this on the publicly available⁷ high-resolution HST-ACS (F606W and F814W) data.

(i) We used the GALAPAGOS-2 software (Häussler et al. 2007) that was developed within the MegaMorph project⁸. GALAPAGOS-2 is an automatic way to provide the morphological parameters with SExtractor (Bertin & Arnouts 1996) and GALFIT (Peng et al. 2002). In this work we provide a single-Sérsic (1968) fit to sources detected in HST-ACS-F814W using SExtractor in high-dynamical range (HDR) dual mode. The HDR mode maximizes the detection of faint sources, and the dual mode ensures that we measured the same sources in all the filters. As indicated in Peng et al. (2010), the formal uncertainties from GALFIT are only lower estimates. After simulations and tests of this software, Häussler et al. (2007) provided uncertainty estimates based on the comparison of input and output values as a function of surface brightness. Because we do not provide simulations like this, we cite the formal uncertainties from GALFIT, but we caution that these are the lower estimates. In Fig. 5 we show the uncertainties on effective radius and Sérsic index as a function of surface brightness defined following Häussler et al. (2007). Even if these are the lower limit errors, we can see the expected tendency of errors to increase with surface brightness. The resulting Sérsic index and its uncertainty is shown in Col. 10 of Table B.1.

Fig. 5. Formal uncertainties of Sérsic index and effective radius from GALFIT. As described in the text, these are the lower limit uncertainties. Black dots shows a sample studied in Nadolny et al. (2020), and blue dots and the green square show the Hα sample and the MZR outlier studied in this work, respectively.

(ii) Visual classification was made using MorphGUI, which is a graphic user interface for morphological classification developed by CANDELS (see Kartaltepe et al. 2015). We modified the interface in order to provide additional morphological classes to extend the classical Hubble scheme, to be able to take into account peculiarities found at higher redshifts. Our classification scheme is based on the work of Elmegreen et al. (2007): point-like (PL), spheroidal (Sph), disc (D), spheroidal+disc (Sph+D), tadpole (T), chain (C), clumpy cluster (CC), doubles (DD), and unclassified (Unclass).

The OTELO GTC/OSIRIS data have considerably lower resolution than HST-ACS (0.256 and 0.03 arcsec px⁻¹, respectively), therefore we performed the match of the two catalogues by allowing multiple matches when there was more than one object in the HST-ACS F814W image inside the Kron ellipse for a source detected in an OTELO_deep image. The classification presented in this work refers to the object closest in position (which usually also is the brightest object) detected in the F814W image of HST-ACS. The detailed morphological analysis of OTELO sources up to z = 2 is the scope of a forthcoming paper (Nadolny et al. 2020).

4. Analysis and discussion

4.1. Mass-metallicity relation

In this section we compare our results with the high-z SDSS, GAMA, and VVDS-Deep and Ultra-Deep samples (see Sect. 2.2 for details on the sample selection). Finally, a comparison is made with the local SDSS sample in order to shed light on the possible evolution of the MZR at the low-mass end.

In Fig. 6 we show the local SDSS sample of galaxies up to z  ∼  0.1 together with higher redshift samples up to z  ∼  0.33 from GAMA and SDSS with their respective second-order polynomial fits [12 + log(O/H) = a + bx + cx² where x = log M_*]. Both high redshift samples were selected to be complete in mass and luminosity at a given redshift. It is worth noting that the OTELO sample is three orders of magnitude deeper in stellar mass than the two surveys at similar redshift. This is due to the depth of the OTELO survey (line flux down to 5.6 × 10⁻¹⁹ erg⁻¹ s⁻¹ cm⁻², see Bongiovanni et al. 2020) and its potential as a blind spectroscopic survey to pre-select ELSs according to their PS and not only as a result of the colour excess in the colour-magnitude diagram. In Fig. 6 we also show the selected sample from the VVDS-Ultra-Deep survey: after our selection process (see Sects. 2.2 and 3.2), we are left with 10 sources from a total of 1125 in the redshift range 0.2 <  z <  0.42 with z_mean = 0.26, which is much wider than our SFG sample redshift range (0.36 <  z <  0.42). The highest redshift in the selected VVDS-Ultra-Deep sample is z  ∼  0.33. The VVDS-Ultra-Deep survey data were processed in a consistent way with our data: (i) Hα fluxes were corrected for stellar absorption (Eq. (4)) and for intrinsic extinction (in this case, using the Balmer decrement); (ii) we estimated the N2 index, SFR (assuming a Chabrier IMF) and sSFR (using VVDS stellar masses), and (iii) we removed sources with N2 > −0.4 as possible AGNs. We note that this sample spans the same range of metallicities, but covers a smaller range in stellar masses. We argue that this shows the capacity of the OTELO survey to recover the low-mass end of the observed galaxy population, even with its limited area of ∼56 arcmin² and despite the limited spectral range (230 Å wide, centred at 9175 Å).

Fig. 6. Mass-metallicity relation for OTELO Hα ELS. Blue and golden markers represent sources with an [N II]λ6583 line flux above and below OTELO flux limit, respectively. The light green empty square shows the MZR outlier as defined by Peeples et al. (2008). Cyan triangles show the VVDS-Ultra-Deep selected sample with zmean = 0.26. Grey filled contours show local SDSS SFGs. Green filled contours show the high-z SDSS sample. Red filled contours show the SFG sample from the GAMA survey. All fits shown in this figure represent second-order polynomial fits. The blue dot-dashed line is the OTELO MZR fit to the sources with an [N II] line flux above the OTELO line flux limit, and the golden dot-dashed line shows the OTELO MZR including low-flux sources; the black thin dashed line shows the fit to the local SDSS sample; and green and red dashed lines show the SDSS and GAMA samples at z  ∼  0.3, respectively. The error bars correspond to the 25% (red) and 68% (black) confidence interval around the most probable value. The calibration uncertainty in metallicity is not included.

In order to shed light on the possible evolution of the MZR, we compared our result with the local SDSS sample. The fit from Tremonti et al. (2004) is valid in the range of 8.5 <  log M_* <  11.5, while our sample reaches masses as low as log M_* ∼ 6.8 (where 63% of the sample with log M_* <  9), therefore we did not use it. In Fig. 6 we show our second-order polynomial fit to the local SDSS sample. From our SFG sample we identify sources with [N II]λ6583 flux below the OTELO line flux limit (low-flux sources). Because for these low-flux sources our metallicity estimate has a large uncertainty, we provide two MZR fits: the first is obtained using sources with an [N II]λ6583 flux that is higher than the OTELO line flux limit (points and dot-dashed line in blue), and the second includes low-flux sources (points and dot-dashed line in gold). For the two fits we limited the c parameter to be not greater than the c parameter form fit of the local SDSS sample. Additionally, we show in this figure the MZR second-order polynomial fits to the local (z <  0.1) and high-z (∼0.31) SDSS and GAMA (z ∼ 0.32) samples.

Our sample lies within the distribution of local SDSS MZR. Although our fit to sources whose [N II]λ6583 line flux exceeds the OTELO line flux limit is flatter at the low-mass end, we cannot conclude that there is an evolution of the MZR because of the uncertainty in the metallicity of the low-flux sources. In the cases of the fit that include low-flux sources (golden symbols), our fit approximates to the local SDSS MZR, especially at the low-mass end. All the fitted polynomial coefficients are listed in Table 2.

In Fig. 6 the upper histogram represents the mass distribution where the red dashed line shows the pure SFG sample and the grey filled histogram also includes possible AGNs (which are not included in the MZR). The “gap” in the M_* distribution between ∼10^8.5 and 10⁹ M_⊙ is a statistical effect due to the low number of sources in the sample. The right-hand histogram shows the metallicity distribution. The peak around ∼7.6 is associated with low-flux sources with an [N II]λ6583 line flux that is lower than the OTELO flux limit. This is the hard limit of our method when it recovers line fluxes from PS, and also for the use of N2 as a metallicity estimate.

4.2. Mass, metallicity, and SFR

As proposed by Lara-López et al. (2010b), the stellar mass, metallicity, and SFR are correlated and may be reduced to a plane in 3D space. In this section we study the scaling relationships between SFR, sSFR, M_*, and metallicity, shown in Fig. 7, where we represent the selected sub-samples from local SDSS, GAMA, VVDS (Deep and Ultra-Deep), and our Hα SFG sample. The left panel of this figure shows the dependence of the SFR on the stellar mass. Noeske et al. (2007) explored this relation over a wide range of redshifts between 0.2 <  z <  1.1, with the mass complete down to ∼10¹⁰. They found a main sequence of this relation for SFGs over cosmic time with a slope of 0.67. Our sample reaches three orders of magnitude lower in stellar mass than the work of Noeske et al. (2007). We find a linear relation with a slightly higher slope of 0.78 ± 0.11, but shifted down by ∼0.25 dex. This shift is associated with the different mass ranges in the two datasets. Using data from merged VVDS Deep and Ultra-Deep survey samples, we obtain a linear fit with a slope of 0.61 ± 0.03, shown in the same figure. We note that this fit approximates ours at the high-mass end. Our fit is in agreement with previous studies within the estimated dispersion. We find no evolution of this relation from the local Universe to z = 0.4. Our Hα sample is consistent with the local SDSS sample, as can be seen not only in the spatial distribution, but also in the linear fits of the two samples. Our sample has moderate SFRs, ranging from −2.2 ≲ log(SFR) ≲ 0.2 M_⊙ yr⁻¹. VVDS covers this relation with a similar dispersion from the high-mass (Deep) down to the low-mass (Ultra-Deep) regime; the VVDS-Ultra-Deep sample reaches the mean mass of the OTELO Hα sample (M_* ∼ 10^8.3 M_⊙).

Fig. 7. Star formation rate and sSFR as a funcion of stellar mass and metallicity. In all panels, black markers show the OTELO SFG sample, green empty circles show transitional dwarf galaxies, cyan triangles show VVDS-Ultra-Deep data, blue dots show VVDS-Deep data, and contours in red and grey show the GAMA and local SDSS samples, respectively. Left panel: linear fits to our SFG sample (blue dashed line), VVDS Deep+Ultra-Deep (both selected sub-samples from VVDS, red dashed-line), and the fit from Noeske et al. (2007; black solid line), which is given in the mass range of 1010 − 1011 M⊙, and the black dashed line is an extrapolation of their fit down to 108 M⊙. The grey dashed line shows the linear fit of the local SDSS sample. For clarity we do not plot the estimated dispersions for any fit.

The middle and right panels of this figure show the relationship of sSFR with M_* and metallicity, respectively. There is no clear evidence of evolution as compared with the local SDSS sample. In both cases the sSFR is anti-correlated with M_* and metallicity (middle and right panel, respectively). Furthermore, these relations hold up to z = 0.4 down to the low-mass regime for the OTELO and VVDS samples.

4.3. Mass-size relation

In Fig. 8 we show the M_*-size relation (MSR) for the Hα sample we studied. We estimated the physical half-flux radius r_e using the effective radius from the Sérsic model and z_OTELO (r_e values are given in the last column of Table B.1). From a linear fit we obtain an intercept of −1.41 ± 0.73 and a slope of 0.19 ± 0.09. In this fit we used 13 sources from the SFG sample that had available M_* and r_e measurements. Here we compare our MSR with Ichikawa et al. (2012), who studied galaxies from the Moircs Deep Survey (Kajisawa et al. 2009) in the GOODS-North region at 0.3 < z < 3 and in a mass range of M_* ∼ 10⁸–10¹¹ M_⊙. They found a universal slope of the MSR (independent of redshift and galaxy activity, SFGs or quiescent, for a given mass) of $R\propto M_{*}^{\alpha }$ with α ∼ 0.1 − 0.2. Figure 8 shows a fit from Ichikawa et al. (2012) for colour-selected SFGs at 0.25 <  z <  0.5 with a slope of 0.115, using their Eq. (3) with the parameters given in their Table 1. In Fig. 8 we also show a linear fit from van der Wel et al. (2014) for colour-selected SFGs from the 3D-HST/CANDELS at z ∼ 0.25 (equation and values as given in their Table 1). We found that our results agree with those from Ichikawa et al. (2012). We argue that this is due to the mass range studied in both works.

Fig. 8. Physical half-light radius re as a function of the stellar mass M*. Black empty circles show the Hα sample, and blue markers show SF galaxies (in both cases we show sources for which it was possible to measure both variables). The green square shows the outlier of the MZR. The blue dashed line represents the linear fit in a form y = a + bx to the Hα sample with a = −1.41 ± 0.73 and b = 0.19 ± 0.09. The green dot-dashed line shows the linear fit from Ichikawa et al. (2012) with the logarithmic slope of 0.115. The red dot-dashed line shows the linear fit found by van der Wel et al. (2014) with a slope of 0.25. The green and red shaded regions represent the dispersion of the linear fit as estimated by Ichikawa et al. (2012) and van der Wel et al. (2014), respectively, while the blue shaded region shows the dispersion of our measurement estimated as the median absolute deviation of 0.2.

4.4. Morphology

In Fig. 9 we show the results of our morphological classification using an automatic fitting of a single-Sérsic model and a visual classification. More than a half (12 sources, 57%; see the background histogram) of the full Hα sample are classified as discs with a median Sérsic index $\overline{n}$ of 1.31. Two (∼9.5%) objects are classified as spheroidal with $\overline{n}$ of 2.2. The increase of the Sérsic index from discs to spheroids is expected because early-type galaxies are fitted with higher Sérsic profile than late-type ones (e.g. Vika et al. 2013). The number of sources classified as point-like and tadpole is negligible (one in each class, 9.5%), with $\overline{n}\sim 4.16$ and 0.7, respectively. For five (24%) sources we are not able to provide the classification with either method. No sources are classified in the following morphological types: chain, clumpy-cluster, and doubles.

Fig. 9. Comparison of the visual (x-axis) and automatic (y-axis) morphological classification or our Hα sample. Stars represent the mean value of the Sérsic index $\overline{n}$ per visual type with error bars calculated as absolute standard deviation. The background histogram shows the total number of sources in each visual type. A total of five sources are visually unclassified, even though two of them have a Sérsic model from GALFIT.

4.5. Possible transitional dwarf galaxy

We identified one source as a possible transitional dwarf galaxy, which is the outlier of the MZR (Fig. 6; green square, source id: 3137 shown in Fig. A.3). The selection is based on the cuts in stellar mass log M_* <  9 and metallicity 12 + log(O/H) > 8.6, following Peeples et al. (2008). According to these authors, it is most likely that sources with these characteristics are metal-rich dwarf galaxies. The percentage of metal-rich dwarf galaxies in our SFG sample is ∼6% (one source in our SFG sample) compared with less than 1% in the local SDSS sample selected with the same criteria. Morphologically, this source is visually classified as a disc with a Sérsic index n = 1.25, it is compact (0.7 kpc), and has a close companion (at least in projection). Peeples et al. (2008) studied a sample of 41 local (z <  0.05) metal-rich dwarf galaxies selected from the SDSS survey from Tremonti et al. (2004). They predicted that in order to have such high metallicity, these sources also need to have a lower gas fraction than objects with similar SFR and luminosities. They discussed other possible explanations for this behaviour: environment (interactions), effective yields, and finally the possibility that these sources are transitional dwarf galaxies at the end of their period of star formation. The outlier in our sample has a low stellar-mass log M_* = 8.2, high metallicity 12 + log(O/H) = 8.6, low SFR of logSFR ∼ −1 M_⊙ yr⁻¹, and a high sSFR of logsSFR ∼ −9.2. More information, in particular spectroscopic observations, is needed to disentangle its morphology and to shed light on its real nature.

5. Conclusions

We studied the MZR and scaling relations between stellar mass, SFR, and sSFR, and the morphology of the low-mass Hα sample drawn from the OTELO survey at z ∼ 0.4. We reach three orders of magnitude deeper in stellar mass (∼10^6.8 < M_*/M_⊙ < 10¹⁰ with 57% of the sample with M_* <  10⁹ M_⊙) than the SDSS and GAMA samples at similar redshift. The PS (data-product of OTELO survey) gives us the possibility to efficiently select and study the metallicity of these low-mass ELSs. About 95% of our ELS sample was selected from spectral features present in PS through a semi-automatic procedure detailed in Sect. 2. If the colour-excess technique used in classical NB surveys had been used instead, only half of this sample (i.e. necessarily the sources with larger observed EWs) would be recovered. For more details, see OTELO-I and Bongiovanni et al. (2020).

Because of the OTELO instrumental limits [as seen in the N2 index and/or 12 + log(O/H)] and the associated errors in the recovery of the line fluxes and the low number of objects in the sample, we cannot conclude about a possible MZR evolution that would be due to cosmic variance. Furthermore, our results are consistent with those obtained using the local SDSS sample (see Fig. 6).

We also explored the SFR as a function of M_*, which has previously been studied by Noeske et al. (2007), who found the so-called main sequence. We showed that this relation holds for the sample we studied, but with a systematic shift by ∼0.25 dex in SFR, which is associated with the different stellar mass ranges used in Noeske et al. (2007) and in this work. Our data agree with the VVDS-Deep and Ultra-Deep data used in this work. Furthermore, the global distribution of the OTELO SFG sample lies within the local SDSS distribution.

From the automatic and visual morphological classification (see Sect. 4.4), we find that the majority of sources in the sample are discs (57%), followed by spheroids (9.5%) with median Sérsic indices of 1.31 and 2.2, respectively. Two sources are classified as point-like and tadpole (one in each class, 9.5% in total). No classification is given for 24% of our sample because they are not detected in the high-resolution HST-ACS F814W image, have no data, or because the sources are just too faint to provide any reliable classification.

We obtained the M_*–size relation for our Hα sample. Our results agrees with those of Ichikawa et al. (2012), but have a slightly higher slope (0.14 and 0.115, respectively). The difference in the results in this work and in van der Wel et al. (2014) is probably due to the different range of stellar-masses that is covered in both studies (we reach almost three orders of magnitude deeper in M_* than van der Wel et al. 2014).

We identified one candidate (∼6% of our SFG sample) as a transitional dwarf galaxy with low mass log M_* = 8.6 M_⊙ and high metallicity 12 + log(O/H) = 8.64. Considering the same criteria, we find less than 1% of these sources in the local-SDSS sample. Morphologically, this source is classified as a compact (∼0.7 kpc) disc galaxy (n = 1.25) with blue (g − i) colour. The study of MZR, SFR, sSFR, and its colour and morphology suggest that this source is indeed a transitional dwarf galaxy at the end of its star formation period, as discussed in previous works (e.g. Peeples et al. 2008). More information is needed to conclude on this issue, however.

Acknowledgments

This work was supported by the project Evolution of Galaxies, of reference AYA2014-58861-C3-1-P, AYA2017-88007-C3-1-P and AYA2017-88007-C3-2-P, within the “Programa estatal de fomento de la investigación científica y técnica de excelencia del Plan Estatal de Investigación Científica y Técnica y de Innovación (2013-2016)” of the “Agencia Estatal de Investigación del Ministerio de Ciencia, Innovación y Universidades”, and co-financed by the FEDER “Fondo Europeo de Desarrollo Regional”. APG and MC are also supported by the Spanish State Research Agency grant MDM-2017-0737 (Unidad de Excelencia María de Maeztu CAB). JAD is grateful for the support from the UNAM-DGAPA-PASPA 2019 program, and the kind hospitality of the IAC. Based on observations made with the Gran Telescopio Canarias (GTC), installed in the Spanish Observatorio del Roque de los Muchachos of the Instituto de Astrofísica de Canarias, on the island of La Palma. MALL is a DARK-Carlsberg Foundation Fellow (Semper Ardens project CF15-0384). This research uses data from the VIMOS VLT Deep Survey, obtained from the VVDS database operated by Cesam, Laboratoire d’Astrophysique de Marseille, France. Based on observations obtained with MegaPrime/MegaCam, a joint project of CFHT and CEA/IRFU, at the Canada-France-Hawaii Telescope (CFHT) which is operated by the National Research Council (NRC) of Canada, the Institut National des Science de l’Univers of the Centre National de la Recherche Scientifique (CNRS) of France, and the University of Hawaii. This work is based in part on data products produced at Terapix available at the Canadian Astronomy Data Centre as part of the Canada-France-Hawaii Telescope Legacy Survey, a collaborative project of NRC and CNRS. This work had made extensive use of TOPCAT and STILTS software (Taylor 2005).

References

Appendix A: Inverse deconvolution and uncertainty estimates

Throughout this paper we made use of the inverse deconvolution of the pseudo-spectra (PS). In this appendix we describe how this deconvolution was performed.

For a given PS we obtain (a) the maximum value of the PS $f_{\mathrm{PS}}^{\max }$ and it observed error ${\sigma }_{\mathrm{PS}}^{\max }$, and (b) an estimate of the continuum level defined as the median flux of the PS f_c and the deviation σ_med, c around this value (mean of the square root of the differences around f_c). We also have a redshift guess value z_GUESS that is obtained from visual inspection of the PS in our GUI web-based tool, and it is assumed to be accurate up to the third digit, that is, with an uncertainty Δz = 0.001. Finally, we define the wavelength region of the PS where the lines can be detected in such a way that it contains at least 2.5 slices (15 Å being 6 Å the average width of the RFT) below the redshifted [N II]λ6548 line and 2.5 slices above the redshifted [N II]λ6583 line:

$$\begin{array}{cc}\hfill {\lambda }_{\min }& =6548·(1+z_{\mathrm{GUESS}}-\mathrm{\Delta }z)-15\hfill \\ \hfill {\lambda }_{\max }& =6583·(1+z_{\mathrm{GUESS}}+\mathrm{\Delta }z)+15.\hfill \end{array}$$(A.1)

We assumed a model spectrum as a rest-frame spectrum defined by Gaussian profiles of the [N II]λ6548, Hα, and [N II]λ6583 lines defined by their amplitude and a common line width for three lines, and a constant continuum level. The model to be compared with the observational data is therefore defined by following parameters: f_mod(z, f_c, σ_line, f_{N II λ6548}, f_Hα, f_{N II λ6583}). We note that there is an intrinsic assumption that this type of model is correct and that there is no additional component in the PS. In two cases (source id: 625 in Fig. A.1, and id: 8074 in Fig. A.2) the PS show Hα + [N II] and [S II]λλ6716,6731 lines. For these we masked the points that correspond to [S II]λλ6716,6731 emission lines. In other cases (source id: 4893 in Fig. A.4) the PS showed an additional component (probably an artefact) that was also removed.

Fig. A.1. Observed PS, the best-fit PS, and the envelope of the PS simulations where all parameters are in the 25% confidence interval (light green) and 68% confidence interval (light grey) for the first eight SFG for which an Hα + [N II] line is detected. Violet, blue, and cyan vertical lines in the PS figures show the position of the [N II]λ6548, Hα, and [N II]λ6583 line, respectively, for the best z obtained in by the fit. The red points that appear in some observed PS were removed during the fitting process because they correspond to the [S II]λλ6716,6731 doublet or may be artefacts. To the right of each PS is the PDF of 12 + log(O/H) that was obtained from the Monte Carlo simulations where the mode (our reference value), the 25%, 68%, and 90% confidence intervals around the mode are shown as vertical lines; the black vertical line in these plots shows the empirical division between SFR and AGN at 12 + log(O/H) = 8.67 (N2 = −0.4).

Fig. A.2. As Fig. A.1 for the remaining two sources with a detected [N II] line.

We performed 10⁶ Monte Carlo simulations over the model parameter space using values of z_GUESS, f_c, σ_med, c, $f_{\mathrm{PS}}^{\max }$, and ${\sigma }_{\mathrm{PS}}^{\max }$ to constrain the parameter space (see below). Each simulation was convolved with the RFT filter system, and a synthetic PS f_PSmod, i for each of the N slices of the PS was obtained. This synthetic PS was compared with the observed PS, and the following likelihood function was obtained for each model and was compared with each point i in the PS:

$$\begin{array}{c}\hfill \mathcal{L}(f_{\mathrm{mod}}|f_{\mathrm{PS}})\propto {\mathrm{\Pi }}_{i=1}^N{w}_{i}exp(-\frac{{\chi }_i^2}{2}),\end{array}$$(A.2)

where χ² is given in the usual way,

$$\begin{array}{c}\hfill {\chi }_i^2={\left(\frac{f_{\mathrm{mod},i}-f_{\mathrm{obs},i}}{{\sigma }_{\mathrm{obs},i}}\right)}^2,\end{array}$$(A.3)

and w_i is a normalized weight function defined as

$$\begin{array}{c}\hfill w_i=\{\begin{array}{cc}\mathrm{MAX}(1,\frac{f_{\mathrm{obs},i}}{f_{\mathrm{c},\mathrm{mod}}})\hfill & \mathrm{if}{\lambda }_i\in [{\lambda }_{\min },{\lambda }_{\max }]\hfill \\ 1\hfill & \mathrm{otherwise}.\hfill \end{array}\end{array}$$(A.4)

The use of the weight function aims to give a greater weight in the fit to the points where the line is presented and are above the continuum level, and in particular, to fit the core of the Hα line. In practical cases, the weight function only has an effect in the objects with faint lines.

In order to sample the likelihood function, the first 30% (300 000) of the simulations made use of the following priors:

1. The redshift z follows a flat distribution in the range z_GUESS ± 0.001.

2. The dispersion of the line, σ_line, varies following a flat distribution from 20 to 500 km s⁻¹, which is translated into a range of 0.43−11 Å for the Hα wavelength. This value is used for the three lines.

3. The continuum value follows a flat distribution in the interval [MAX(0, f_c − σ_med, c),f_c + σ_med, c].

4. The rest-frame f_Hα line flux varies following a flat distribution in the interval [f_c, f_max], where f_max is given by

   $$\begin{array}{cc}\hfill f_{\max }=& (f_{\mathrm{med},\mathrm{c}}+f_{\max }^{\mathrm{PS}}+3\sigma (f_{\mathrm{PS}}^{\max }))·\hfill \\ \hfill & {\sigma }_{\mathrm{line}\max }·(1+z_{\max })·\sqrt{2\pi }.\hfill \end{array}$$(A.5)

5. The rest-frame f_{N II λ6583} line flux varies following a flat distribution between 0.006 ⋅ f_Hα and f_max.

6. The f_{N II λ6548} line flux is fixed to 1/3 ⋅ f_{N II λ6583}.

When these 300 000 simulations were obtained, we computed a preliminary likelihood function. This likelihood was sampled in a regular interval with 200 bins over the range in which each parameter varied.

This likelihood function was marginalized for each parameter (i.e. we made a projection of the PDF on each particular parameter and the corresponding PDF was normalized). We obtained the marginalized likelihood using a regular interval of 200 bins over the range in which each parameter varied, and we obtained the range that contains 99% of the simulations around the value with the maximum likelihood. The next 20% (200 000) of the simulations were restricted to the parameters that sample this 99% confidence interval. This process was repeated five more times (using 10% (100 000) of the simulations at each step) with the limits obtained by the 85%, 75%, 68%, 40%, and 10% confidence intervals around the most probable value.

After this process we had a final PDF shape for each parameter, as well as all the required additional quantities such as the EWs, the N2 parameter, or the metallicities. We note that the asymmetry of the PDF implies that the mean and the standard deviation (squared of the variance) are not informative to describe the distribution. For this reason we used the mode of the resulting PDF as the reference value, and we obtained the region around the mode that included 10%, 25%, 50%, 68%, and 90% of the area. In the main text the error bars show the 25% and 68% confidence interval, which also gives an idea of the asymmetry of the associated distribution.

We show in Figs. A.1 and A.2 the observed PS, the best-fit PS, and the envelope of the PS simulations where all parameters are in the 25% confidence interval (light green) and 68% confidence interval (light grey) for the SFG for which the Hα + [N II] line was detected. We also show the PDF of 12 + log(O/H) that we obtained from the Monte Carlo simulations for each source where the mode (our reference value), the 25%, 68%, and 90% confidence intervals around the mode are shown in vertical lines. We also show with a black vertical line the empirical division between SFG and AGN at 12 + log(O/H) = 8.67 (N2 = −0.4). Figures A.3 and A.5 show similar information for the possible transitional dwarf galaxy id: 3137, and for sources whose [N II] line flux is lower than the OTELO line flux limit (low-flux sources), respectively; we note that for the latter, the PDF of 12 + log(O/H) has a step-like shape that reflects the large uncertainty in the [N II] line flux. Finally, Fig. A.4 show the same information for AGN sources; we kept the PDF of 12 + log(O/H) for comparison with the previous plots.

Fig. A.3. As Fig. A.1, but for the possible transitional dwarf galaxy (source id: 3137).

Fig. A.4. As Fig. A.1, but for the three AGN candidates.

Fig. A.5. As Fig. A.1, but for the five sources whose [N II] line flux is lower than the OTELO line flux limit (low-flux sources).

We show in Table B.1 the modal values and 68% confidence intervals around the mode for f_Hα and EW(Hα) (after correction for stellar absorption), f_{N II λ6583}, and 12 + log(O/H). See Sect. 3.2 for details about the uncertainties of z_OTELO.

Appendix B: Uncertainty in stellar masses and SFR

As explained in Sect. 3, stellar masses were computed using the prescription of LS18, which makes use of colour (g − i) and the i absolute magnitude. The associated uncertainty was computed making use of the R subroutine propagate, which makes an estimate of the error by a first- and second-order propagation of any input recipe, as well as by Monte Carlo simulations. The uncertainty quoted in Table B.1 corresponds to the variance obtained from 10 000 the Monte Carlo simulations using that sub-routine.

Finally, the uncertainty in the SFR quoted in Table B.1 requires some cautions. First of all, the SFR requires the use of an extinction value. We used the value that was obtained with the best-fit solution with the LePhare software and used empirical galaxy templates. This choice implies that the extinction value is a lower limit because the intrinsic extinction of the templates are unknown. In addition, LePhare does not provide an evaluation of the uncertainty in E(B − V), therefore this uncertainty was not included in the SFR. Finally, we recall that our f_Hα uncertainty does not correspond to a variance (where standard propagation theory would apply), but to a confidence interval. Taking these considerations into account, we computed the uncertainty in the SFR by applying the same formulae as we used to obtain the nominal SFR (the extinction correction used by Ly et al. 2012 and the SFR estimate using the Kennicutt & Evans 2012 relation) to the f_Hα values that define the 68% confidence interval. We stress that the uncertainty values would be underestimated because they do not include all the possible sources of uncertainty (interstellar extinction in particular).

All Tables

All Figures

Fig. 1. Pseudo-spectra of selected sources. From left: source id: 2146 with a clear Hα + [N II] emission line complex at z = 0.387; source id: 625 with Hα (truncated in the blue side) and [N II] emission lines and [S II]λ6716 (on the red side of the spectrum) at z = 0.371; and source id: 3289, which exhibits the [S II]λλ6716,6731 doublet at z = 0.340. The green line shows the best-fit synthetic spectrum after the convolution process. See Sect. 3.2 for details.

In the text

	Fig. 2. Comparison of computed stellar masses M* using prescriptions from López-Sanjuan et al. (2019) and Taylor et al. (2011). The overall OTELO and Hα sample are shown as black dots and black empty circles, respectively. Dashed-line in blue represent 1:1 line.
In the text

	Fig. 3. Colour–M* diagram. Blue points show Hα ELS, and green contours represent the overall OTELO sample. Filled contours in grey show the envelope of SDSS-DR7 data, and filled red and blue contours show the red and blue cloud separated with the empirically estimated limit of Bluck et al. (2014). Green squares mark MZR outlier galaxies (see Sect. 4.1).
In the text

Fig. 4. N2 index vs. EWHα. Blue points show all the Hα candidates. Black and golden circles represent sources with an [N II] line flux above and below OTELO flux limit, respectively (see Sect. 3.2 and Bongiovanni et al. 2020). Red circles show AGN candidates selected using the Stasińska et al. (2006) empirical limit of N2 = −0.4. Vertical green and blue dashed lines show the Yin et al. (2007) and Pettini & Pagel (2004) limits of their calibration using the N2 index, respectively. The red line shows the AGN selection limit from Stasińska et al. (2006). The error bars in each object correspond to the 25% and 68% confidence interval around the most probable value.

In the text

	Fig. 5. Formal uncertainties of Sérsic index and effective radius from GALFIT. As described in the text, these are the lower limit uncertainties. Black dots shows a sample studied in Nadolny et al. (2020), and blue dots and the green square show the Hα sample and the MZR outlier studied in this work, respectively.
In the text

Fig. 6. Mass-metallicity relation for OTELO Hα ELS. Blue and golden markers represent sources with an [N II]λ6583 line flux above and below OTELO flux limit, respectively. The light green empty square shows the MZR outlier as defined by Peeples et al. (2008). Cyan triangles show the VVDS-Ultra-Deep selected sample with zmean = 0.26. Grey filled contours show local SDSS SFGs. Green filled contours show the high-z SDSS sample. Red filled contours show the SFG sample from the GAMA survey. All fits shown in this figure represent second-order polynomial fits. The blue dot-dashed line is the OTELO MZR fit to the sources with an [N II] line flux above the OTELO line flux limit, and the golden dot-dashed line shows the OTELO MZR including low-flux sources; the black thin dashed line shows the fit to the local SDSS sample; and green and red dashed lines show the SDSS and GAMA samples at z  ∼  0.3, respectively. The error bars correspond to the 25% (red) and 68% (black) confidence interval around the most probable value. The calibration uncertainty in metallicity is not included.

In the text

Fig. 7. Star formation rate and sSFR as a funcion of stellar mass and metallicity. In all panels, black markers show the OTELO SFG sample, green empty circles show transitional dwarf galaxies, cyan triangles show VVDS-Ultra-Deep data, blue dots show VVDS-Deep data, and contours in red and grey show the GAMA and local SDSS samples, respectively. Left panel: linear fits to our SFG sample (blue dashed line), VVDS Deep+Ultra-Deep (both selected sub-samples from VVDS, red dashed-line), and the fit from Noeske et al. (2007; black solid line), which is given in the mass range of 1010 − 1011 M⊙, and the black dashed line is an extrapolation of their fit down to 108 M⊙. The grey dashed line shows the linear fit of the local SDSS sample. For clarity we do not plot the estimated dispersions for any fit.

In the text

Fig. 8. Physical half-light radius re as a function of the stellar mass M*. Black empty circles show the Hα sample, and blue markers show SF galaxies (in both cases we show sources for which it was possible to measure both variables). The green square shows the outlier of the MZR. The blue dashed line represents the linear fit in a form y = a + bx to the Hα sample with a = −1.41 ± 0.73 and b = 0.19 ± 0.09. The green dot-dashed line shows the linear fit from Ichikawa et al. (2012) with the logarithmic slope of 0.115. The red dot-dashed line shows the linear fit found by van der Wel et al. (2014) with a slope of 0.25. The green and red shaded regions represent the dispersion of the linear fit as estimated by Ichikawa et al. (2012) and van der Wel et al. (2014), respectively, while the blue shaded region shows the dispersion of our measurement estimated as the median absolute deviation of 0.2.

In the text

Fig. 9. Comparison of the visual (x-axis) and automatic (y-axis) morphological classification or our Hα sample. Stars represent the mean value of the Sérsic index $\overline{n}$ per visual type with error bars calculated as absolute standard deviation. The background histogram shows the total number of sources in each visual type. A total of five sources are visually unclassified, even though two of them have a Sérsic model from GALFIT.

In the text

Fig. A.1. Observed PS, the best-fit PS, and the envelope of the PS simulations where all parameters are in the 25% confidence interval (light green) and 68% confidence interval (light grey) for the first eight SFG for which an Hα + [N II] line is detected. Violet, blue, and cyan vertical lines in the PS figures show the position of the [N II]λ6548, Hα, and [N II]λ6583 line, respectively, for the best z obtained in by the fit. The red points that appear in some observed PS were removed during the fitting process because they correspond to the [S II]λλ6716,6731 doublet or may be artefacts. To the right of each PS is the PDF of 12 + log(O/H) that was obtained from the Monte Carlo simulations where the mode (our reference value), the 25%, 68%, and 90% confidence intervals around the mode are shown as vertical lines; the black vertical line in these plots shows the empirical division between SFR and AGN at 12 + log(O/H) = 8.67 (N2 = −0.4).

In the text

	Fig. A.2. As Fig. A.1 for the remaining two sources with a detected [N II] line.
In the text

	Fig. A.3. As Fig. A.1, but for the possible transitional dwarf galaxy (source id: 3137).
In the text

	Fig. A.4. As Fig. A.1, but for the three AGN candidates.
In the text

	Fig. A.5. As Fig. A.1, but for the five sources whose [N II] line flux is lower than the OTELO line flux limit (low-flux sources).
In the text