TY  - JOUR
A1  - Lötsch, Jörn
A1  - Mayer, Benjamin
T1  - A biomedical case study showing that tuning random forests can fundamentally change the interpretation of supervised data structure exploration aimed at knowledge discovery
T2  - BioMedInformatics
N2  - Knowledge discovery in biomedical data using supervised methods assumes that the data contain structure relevant to the class structure if a classifier can be trained to assign a case to the correct class better than by guessing. In this setting, acceptance or rejection of a scientific hypothesis may depend critically on the ability to classify cases better than randomly, without high classification performance being the primary goal. Random forests are often chosen for knowledge-discovery tasks because they are considered a powerful classifier that does not require sophisticated data transformation or hyperparameter tuning and can be regarded as a reference classifier for tabular numerical data. Here, we report a case where the failure of random forests using the default hyperparameter settings in the standard implementations of R and Python would have led to the rejection of the hypothesis that the data contained structure relevant to the class structure. After tuning the hyperparameters, classification performance increased from 56% to 65% balanced accuracy in R, and from 55% to 67% balanced accuracy in Python. More importantly, the 95% confidence intervals in the tuned versions were to the right of the value of 50% that characterizes guessing-level classification. Thus, tuning provided the desired evidence that the data structure supported the class structure of the data set. In this case, the tuning made more than a quantitative difference in the form of slightly better classification accuracy, but significantly changed the interpretation of the data set. This is especially true when classification performance is low and a small improvement increases the balanced accuracy to over 50% when guessing.
KW  - data science
KW  - artificial intelligence
KW  - machine-learning
KW  - digital medicine
Y1  - 2022
UR  - http://publikationen.ub.uni-frankfurt.de/frontdoor/index/index/docId/75564
UR  - https://nbn-resolving.org/urn:nbn:de:hebis:30:3-755644
SN  - 2673-7426
VL  - 2.2022
IS  - 4
SP  - 544
EP  - 552
PB  - MDPI
CY  - Basel
ER  -