Metrics on the test set of stars from the m12i LSR0 FIRE mock catalog with v_los measurements and δϖ/ϖ < 0.10. Such ROC curves compare the accreted and in situ efficiencies, as defined in Eq. (2), for different networks. In general, a network has better performance if the accreted efficiency is high, and the in situ efficiency is low, that is, if the ROC curve tends to the top left of a given plot. In this figure, the label on each curve denotes what information the network had access to at training, and the plot title indicates the common network input for all curves in that plot. Note that the same nine curves are plotted in the top row and the bottom row – they are only organized differently to make the comparisons as transparent as possible. The gray symbols denote the performance for the V and VM selections described in Sect. 3. ZM is out of the range of the plots, as it has an in situ efficiency of 1.8 × 10⁻⁴ and an accreted efficiency of 3.3 × 10⁻². The network performance consistently beats the traditional methods in all cases where metallicity is included as an input, even if only 4D kinematics are available. The addition of photometric inputs provides a relative improvement compared to kinematics-only networks, although it is not as powerful as metallicity.