New AI Benchmark Tests Machines' Ability to Resolve Ambiguous Sentences Using Visual Clues
Researchers created LaViSA, a benchmark designed to test whether AI models can resolve structurally ambiguous sentences by leveraging visual scenes. This helps machines better interpret complex, real-world situations where word order alone isn't enough.
Researchers introduced LaViSA (Language and Vision Structural Ambiguity), a new benchmark to evaluate how well Vision and Language Models (VLMs) can resolve structural ambiguity in sentences using visual scenes. Structural ambiguity occurs when a single sentence has multiple valid interpretations due to its syntactic structure. For example, in the sentence "I saw the man on the hill with a telescope," the phrase "with a telescope" could logically modify either "the man" (the man has a telescope) or "the hill" (the hill has a telescope on it), or even the act of seeing. LaViSA tests whether VLMs can correctly leverage corresponding visual cues to determine the intended meaning.
This benchmark matters because understanding such ambiguities is crucial for practical AI applications like conversational assistants, autonomous navigation, and image retrieval. In a real-world scenario, an AI assistant that correctly interprets visual context—such as knowing whether an object belongs to a person or a location—can provide more accurate responses.
Researchers used the benchmark to evaluate several state-of-the-art VLMs, and initial results reveal that even advanced models struggle with certain types of structural ambiguity, especially when subtle visual differences distinguish between interpretations. This points to a gap in current machine understanding of both language and vision.