Continuous Value Tokenization Improves Medical Event Foundation Models

Medical foundation models convert patient records into token sequences for autoregressive prediction, but numeric values such as lab results, vital signs, and time intervals are typically discretized into bins, losing precision and misaligning with clinical thresholds. We trained decoder-only transformer models (47 million parameters) on MIMIC-IV data (364,627 patients; 375 million observations) to compare three tokenization strategies: Discrete (binned values), Continuous Factored (continuous values preserving sequence length), and Continuous Fused (continuous values fused with measurement-type tokens). We evaluated next-token prediction, numeric value prediction, and three clinical tasks: ED disposition at triage, ICD code prediction, and DRG prediction at discharge. Continuous Fused tokenization reduced median sequence length by 34\%, reached the Discrete model's final next-token loss in 30\% of training iterations, and improved numeric prediction accuracy by 30.25\% median nRMSE reduction. ICD code prediction favored Continuous Fused (AU-PRC 0.457 vs.\ 0.446; p < 0.001); DRG prediction was equivalent between Continuous Fused and Discrete; ED disposition accuracy was equivalent across all models ($\sim$0.900), though Discrete achieved better calibration. We additionally explain why predictive performance improves with Monte Carlo sample count and derive a scaling law to predict performance gains from increasing simulation budget. Continuous-value tokenization offers substantial efficiency and precision gains while maintaining comparable clinical task performance, with no modifications to the standard transformer architecture.