Receiver operating characteristic curve for planets in all three models. The ROC metric varies the class probability threshold and compares, at each new value, the recall of a particular model (or true positive rate) with its false-positive rate. The curve drawn then allows comparison of classification rates even across models trained with different balances of planets and false positives. Random guessing would produce a diagonal line through the origin to [1,1], which would not be visible on this zoomed figure. Perfect models lie as close as possible to the top-left corner. We show mean-averaged ensemble models for the three model types (binary, three-class and four-class models). We also plot “multimodel” averages which produce an ensemble of the estimated planet probabilities from each sample in all three of those models. The area under each curve (ROC-AUC), a statistic which corresponds to the probability that the correct class outranks the incorrect one, is also shown for each model in the legend.