Restrepo, FelipeMali, NamrataAbrahams, AlanRactham, Peter2023-06-062023-06-062022-072046-1402PMC9350436http://hdl.handle.net/10919/115348Conventional binary classification performance metrics evaluate either general measures (accuracy, F score) or specific aspects (precision, recall) of a model's classifying ability. As such, these metrics, derived from the model's confusion matrix, provide crucial insight regarding classifier-data interactions. However, modern- day computational capabilities have allowed for the creation of increasingly complex models that share nearly identical classification performance. While traditional performance metrics remain as essential indicators of a classifier's individual capabilities, their ability to differentiate between models is limited. In this paper, we present the methodology for MARS (Method for Assessing Relative Sensitivity/ Specificity) ShineThrough and MARS Occlusion scores, two novel binary classification performance metrics, designed to quantify the distinctiveness of a classifier's predictive successes and failures, relative to alternative classifiers. Being able to quantitatively express classifier uniqueness adds a novel classifier-classifier layer to the process of model evaluation and could improve ensemble model-selection decision making. By calculating both conventional performance measures, and proposed MARS metrics for a simple classifier prediction dataset, we demonstrate that the proposed metrics' informational strengths synergize well with those of traditional metrics, delivering insight complementary to that of conventional metrics.application/pdfenCreative Commons Attribution 4.0 InternationalBinary classificationClassifier comparative uniquenessClassifier performance evaluationClassifier selection optimizationMachine learningSensitivity and SpecificityArticle - Refereed2023-06-06https://doi.org/10.12688/f1000research.110567.211Abrahams, Alan [0000-0002-4884-3192]359679702046-1402