Study Reveals Why LLMs Behave Unpredictably Based on Prompts

A new study published on arXiv (2604.22027) investigates why large language models (LLMs) often produce inconsistent results based on how a question is phrased. The research compares two common prompting methods: instruction-based prompts, which describe the task in natural language, and example-based prompts, which use in-context few-shot demonstrations. The findings suggest that despite the differences in prompt style, LLMs rely on shared lexical task representations, leading to variability in responses.

This research is significant because it provides a deeper understanding of why LLMs can be so sensitive to prompt phrasing. The study highlights that the underlying representations of tasks in LLMs are not as stable as previously thought, which can lead to unpredictable behavior. This insight could help developers create more consistent and reliable LLM outputs by better understanding how task representations are formed and utilized.

The implications of this study are far-reaching. If developers can identify and stabilize these shared task representations, they may be able to reduce the variability in LLM responses. Future research could explore techniques to make these representations more robust, potentially leading to more consistent and reliable AI systems. The study also opens up questions about how different prompting strategies might interact with these representations and whether there are optimal ways to structure prompts to minimize variability.