TELF: An End-to-End Temporal Encoder with Late Fusion for Interpretable Disease Risk Prediction from Longitudinal Real-World Data

Deep learning models utilizing longitudinal healthcare data have significantly advanced epidemiological research. However, contemporary transformer-based models increasingly rely on computationally intensive pre-training steps that entail processing massive real-world datasets with cost-prohibitive hardware. We introduce the Temporal Encoder with Late Fusion (TELF), a lightweight end-to-end predictive model featuring an encoder-only architecture for processing medical codes, followed by post-encoder concatenation with demographic variables. TELF learns code embeddings on-the-fly, thereby bypassing the resource-intensive pre-training bottleneck. Furthermore, its late-fusion design preserves the integrity of the temporal attention mechanism before integrating static demographic predictors. We evaluated TELF using an administrative claims database across three distinct cohorts: pancreatic cancer (n=53,661), type 2 diabetes (n=78,756), and heart failure (n=72,540). TELF consistently outperformed traditional machine learning baselines, including XGBoost, LightGBM, and logistic regression. Specifically, TELF achieved AUCs of 0.9150, 0.8199, and 0.8721 for pancreatic cancer, type 2 diabetes, and heart failure, respectively, compared with 0.9044, 0.7908, and 0.8535 for XGBoost and 0.9014, 0.7800, and 0.8466 for logistic regression. Beyond predictive superiority, TELF's isolated temporal attention mechanism enables population-level motif mining. By extracting high-attention temporal sequences, we mapped aggregated patient journey pathways, revealing interpretable clinical trajectories preceding disease onset. Collectively, these results demonstrate that TELF provides a resource-efficient and accessible framework for advanced temporal modeling in clinical and epidemiological research.