Agentic LLM Systems Show Promise in Breast Cancer Treatment Recommendations, Study Finds
A new study evaluated agentic LLM systems on 72 real breast cancer cases using 1,147 rubrics generated via Asymmetric Information Rubric Generation (AIRG). Seven pipelines were compared, with results suggesting AI can assist but not replace human oncologists.
Researchers have evaluated agentic large language model (LLM) systems for generating breast cancer treatment recommendations using 72 real clinical cases spanning stages I to IV. The study, published on arXiv, employed 1,147 case-specific rubrics created through a novel method called Asymmetric Information Rubric Generation (AIRG), where the rubric generator had access to real clinical decisions that the evaluated models did not. Seven different AI pipelines were compared, including approaches based on LLMs and those incorporating specialized medical knowledge.
This matters because breast cancer treatment is complex, and doctors often face difficult decisions under time constraints. AI systems that can quickly analyze a patient's case and suggest evidence-based treatments could help improve decision-making and potentially lead to better patient outcomes. However, the researchers found that while some AI systems performed well, none achieved perfect accuracy. The study underscores that AI is not yet ready to replace human expertise but could serve as a valuable assistive tool in clinical settings.
For those interested in the technical details, the full paper is available on arXiv under the title 'Agentic systems for breast cancer treatment recommendations.'