New Research Shows AI Training Method Hurts Disease Prediction Accuracy

Researchers tested a popular AI training method on predicting Alzheimer's disease and related dementias (ADRD) from longitudinal health histories. Across a large-scale controlled experiment of 504 configurations, they found that supervised fine-tuning with synthetic rationale data (teaching AI models to explain their predictions) consistently and substantially hurt prediction performance compared to label-only fine-tuning. This is surprising because most experts thought this method would improve AI performance in healthcare.

This matters because AI is increasingly used in medical diagnosis. If we train AI models to explain their reasoning, they might make more mistakes. The study suggests that for critical tasks like disease prediction, simple training methods might work better than complex ones that try to mimic human reasoning.