A Robust and Interpretable Feature Engineering Approach for Low-Data Biological Classification
Gora, S.; Dervishi, E.
Show abstract
Accurate biological classification often faces challenges from high-dimensional data and limited samples, leading to model overfitting and poor interpretability. This study introduces Directional Flow Embedding (DFE), a novel feature engineering and embedding method designed to overcome these issues. DFE transforms raw biological data into three concise and biologically interpretable features: Directional Flow Score (DFS), Heterogeneity Index (HI), and Local Density Estimate (LDE). It achieves this by robustly determining a global direction vector from class means, enabling the projection of samples onto this principal axis, quantifying their deviation, and incorporating local density information. Evaluated on real-world TCGA-BRCA and TCGA-LUAD RNA-sequencing datasets, DFE consistently demonstrated superior performance. It significantly outperformed traditional methods and strong non-linear models. Ablation studies confirmed the synergistic contribution of all three DFE features, while sensitivity analysis revealed its robustness. DFEs inherent interpretability, strong generalizability, and computational efficiency make it a valuable tool for robust and transparent biological classification, thereby advancing critical applications in biomedical research and clinical decision-making.
Matching journals
The top 8 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Exploring High-Dimensional Biological Data with Sparse Contrastive Principal Component Analysis 96%
- Adversarial Deconfounding Autoencoder for Learning Robust Gene Expression Embeddings 96%
- High-dimensional Biomarker Identification for Scalable and Interpretable Disease Prediction via Machine Learning Models 96%
Similar papers in this journal
Similar papers in this journal
- Stochastic LASSO for extremely high-dimensional genomic data 95%
- DeepInsight-3D for precision oncology: an improved anti-cancer drug response prediction from high-dimensional multi-omics data with convolutional neural networks 94%
- Generalising uncertainty improves accuracy and safety of deep learning analytics applied to oncology 94%
Similar papers in this journal
- DeepMF: Deciphering the Latent Patterns in Omics Profiles with a Deep Learning Method 95%
- Autoencoders with shared and specific embeddings formulti-omics data integration 95%
- Leveraging Permutation Testing to Assess Confidence in Positive-Unlabeled Learning Applied to High-Dimensional Biological Datasets 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.