Formal definition of the MARS method for quantifying the unique target class discoveries of selected machine classifiers

Simple item page
dc.contributor.authorRestrepo, Felipeen
dc.contributor.authorMali, Namrataen
dc.contributor.authorAbrahams, Alanen
dc.contributor.authorRactham, Peteren
dc.date.accessioned2023-06-06T16:59:23Zen
dc.date.available2023-06-06T16:59:23Zen
dc.date.issued2022-07en
dc.date.updated2023-06-06T14:34:41Zen
dc.description.abstractConventional binary classification performance metrics evaluate either general measures (accuracy, F score) or specific aspects (precision, recall) of a model's classifying ability. As such, these metrics, derived from the model's confusion matrix, provide crucial insight regarding classifier-data interactions. However, modern- day computational capabilities have allowed for the creation of increasingly complex models that share nearly identical classification performance. While traditional performance metrics remain as essential indicators of a classifier's individual capabilities, their ability to differentiate between models is limited. In this paper, we present the methodology for MARS (Method for Assessing Relative Sensitivity/ Specificity) ShineThrough and MARS Occlusion scores, two novel binary classification performance metrics, designed to quantify the distinctiveness of a classifier's predictive successes and failures, relative to alternative classifiers. Being able to quantitatively express classifier uniqueness adds a novel classifier-classifier layer to the process of model evaluation and could improve ensemble model-selection decision making. By calculating both conventional performance measures, and proposed MARS metrics for a simple classifier prediction dataset, we demonstrate that the proposed metrics' informational strengths synergize well with those of traditional metrics, delivering insight complementary to that of conventional metrics.en
dc.description.versionPublished versionen
dc.format.mimetypeapplication/pdfen
dc.identifier.doihttps://doi.org/10.12688/f1000research.110567.2en
dc.identifier.eissn2046-1402en
dc.identifier.issn2046-1402en
dc.identifier.orcidAbrahams, Alan [0000-0002-4884-3192]en
dc.identifier.otherPMC9350436en
dc.identifier.pmid35967970en
dc.identifier.urihttp://hdl.handle.net/10919/115348en
dc.identifier.volume11en
dc.language.isoenen
dc.publisherF1000 Researchen
dc.relation.urihttps://www.ncbi.nlm.nih.gov/pubmed/35967970en
dc.rightsCreative Commons Attribution 4.0 Internationalen
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/en
dc.subjectBinary classificationen
dc.subjectClassifier comparative uniquenessen
dc.subjectClassifier performance evaluationen
dc.subjectClassifier selection optimizationen
dc.subjectMachine learningen
dc.subject.meshSensitivity and Specificityen
dc.titleFormal definition of the MARS method for quantifying the unique target class discoveries of selected machine classifiersen
dc.title.serialF1000Researchen
dc.typeArticle - Refereeden
dc.type.dcmitypeTexten
dc.type.otherResearch Support, Non-U.S. Gov'ten
dc.type.othermethods-articleen
dc.type.otherJournal Articleen
dcterms.dateAccepted2022-06-16en
pubs.organisational-group/Virginia Techen
pubs.organisational-group/Virginia Tech/Engineeringen
pubs.organisational-group/Virginia Tech/Engineering/Industrial and Systems Engineeringen
pubs.organisational-group/Virginia Tech/Pamplin College of Businessen
pubs.organisational-group/Virginia Tech/Pamplin College of Business/Business Information Technologyen
pubs.organisational-group/Virginia Tech/All T&R Facultyen
pubs.organisational-group/Virginia Tech/Pamplin College of Business/PCOB T&R Facultyen
pubs.organisational-group/Virginia Tech/Report testen
pubs.organisational-group/Virginia Tech/Graduate studentsen
pubs.organisational-group/Virginia Tech/Graduate students/Masters studentsen

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Formal definition of the MARS method for quantifying the unique target class discoveries of selected machine classifiers.pdf
Size:
679.56 KB
Format:
Adobe Portable Document Format
Description:
Published version
Download

Collections

All Faculty Deposits
Scholarly Works, Business Information Technology
Scholarly Works, Computer Science
Scholarly Works, Industrial and Systems Engineering