MosaicLeaks: Can your research agent keep a secret?
ServiceNow researchers discovered a vulnerability in AI research agents that can leak sensitive data through a novel attack called MosaicLeaks. The flaw exploits how these agents handle and combine information, potentially exposing private details even when individual queries seem safe. The issue affects multiple AI research platforms and highlights a fundamental security challenge in agentic AI systems.
ServiceNow released a report called MosaicLeaks, revealing a new type of security vulnerability in AI research agents. These agents, designed to help with research by gathering and synthesizing information from various sources, can inadvertently leak sensitive data through a technique the researchers call a "mosaic attack." Unlike a simple data leak, a mosaic attack works by combining seemingly harmless pieces of information from different queries or sources to reconstruct private details that were never meant to be shared.
For example, an AI research agent might be asked to summarize a public dataset and later to draft a report on a specific topic. If the agent retains context across tasks, it could inadvertently stitch together fragments of information—like a name from one query and a medical condition from another—to reveal something sensitive. This is particularly dangerous because the individual queries appear safe, but the aggregated output can expose private data.
The researchers demonstrated this flaw across multiple AI research platforms, showing that the vulnerability is not limited to a single tool or model. It stems from how these agents are designed to maintain context and combine information, a feature that is both their strength and their weakness.
To protect yourself, the researchers recommend using AI research tools that implement strict data isolation between tasks, regularly auditing agent outputs for unintended information leakage, and applying the latest security patches from providers. For users of Hugging Face's ecosystem, the ServiceNow blog post on Hugging Face provides detailed technical findings and mitigation strategies.