Zenity Labs & MITRE ATLAS Collaborate to Advance AI Agent Security with the First Release of Agent-Focused TTPs

TL;DR

Zenity Labs worked in collaboration with MITRE ATLAS to incorporate the first 14 agent-focused techniques and subtechniques, extending the framework beyond LLM threats to cover the unique risks posed by AI agents.

Zenity’s Expertise with AI Agents

At Zenity, we specialize in securing AI agents. Our research explores how agents can be misconfigured, manipulated, and exploited in real-world environments. We have presented our findings at leading security conferences such as Black Hat and AI Agent Security Summit, highlighting the risks and new attack surface introduced by AI agents. This body of research, combined with insights from other published work, forms the basis of the AI Agents Attack Matrix - an open source framework that documents attack techniques that can be used against AI agents. The matrix serves as a practical resource for defenders, helping them understand how adversaries exploit agent capabilities and providing security teams with clarity on where to focus time, tools, and resources.

New Attack Techniques Added to ATLAS

MITRE ATLAS and Zenity have been collaborating to incorporate and adapt techniques from the AI Agents Attack Matrix into MITRE ATLAS. The previous release addressed broad AI-related threats. In this update, we took a major step forward to define and document the distinct risks introduced by autonomous agents.

A total of 14 new techniques and subtechniques were added through this joint collaboration:

• AI Agent Context Poisoning - Adversaries may attempt to manipulate the context used by an AI agent's large language model (LLM) to influence the responses it generates or actions it takes. This allows an adversary to persistently change the behavior of the target agent and further their goals.
• Memory - Adversaries may manipulate the memory of a large language model (LLM) in order to persist changes to the LLM to future chat sessions.
• Thread - Adversaries may introduce malicious instructions into a chat thread of a large language model (LLM) to cause behavior changes which persist for the remainder of the thread. A chat thread may continue for an extended period over multiple sessions.
• Modify AI Agent Configuration - Adversaries may modify the configuration files for AI agents on a system. This allows malicious changes to persist beyond the life of a single agent and affects any agents that share the configuration.
• RAG Credential Harvesting - Adversaries may attempt to use their access to a large language model (LLM) on the victim's system to collect credentials. Credentials may be stored in internal documents which can inadvertently be ingested into a RAG database, where they can ultimately be retrieved by an AI agent.
• Credentials from AI Agent Configuration - Adversaries may access the credentials of other tools or services on a system from the configuration of an AI agent.
• Discover AI Agent Configuration - Adversaries may attempt to discover configuration information for AI agents present on the victim's system. Agent configurations can include tools or services they have access to.
• Embedded Knowledge - Adversaries may attempt to discover the data sources a particular agent can access. The AI agent's configuration may reveal data sources or knowledge.
• Tool Definitions - Adversaries may discover the tools the AI agent has access to. By identifying which tools are available, the adversary can understand what actions may be executed through the agent and what additional resources it can reach. This knowledge may reveal access to external data sources or expose exfiltration paths, helping adversaries identify AI agents that provide the greatest value or opportunity for attack.
• Activation Triggers - Adversaries may discover keywords or other triggers (such as incoming emails, documents being added, incoming message, or other workflows) that activate an agent and may cause it to run additional actions.
• Data from AI Services - Adversaries may use their access to a victim organization's AI-enabled services to collect proprietary or otherwise sensitive information. As organizations adopt generative AI in centralized services for accessing an organization's data, such as with chat agents which can access retrieval augmented generation (RAG) databases and other data sources via tools, they become increasingly valuable targets for adversaries.
• RAG Databases - Adversaries may prompt the AI service to retrieve data from a RAG database. This can include the majority of an organization's internal documents.
• AI Agent Tools - Adversaries may prompt the AI service to invoke various tools the agent has access to. Tools may retrieve data from different APIs or services in an organization.
• Exfiltration via AI Agent Tool Invocation - Adversaries may use prompts to invoke an agent's tool capable of performing write operations to exfiltrate data. Sensitive information can be encoded into the tool's input parameters and transmitted as part of a seemingly legitimate action. Variants include sending emails, creating or modifying documents, updating CRM records, or even generating media such as images or videos.

Incorporating these techniques into MITRE ATLAS extends the framework to address the unique threats posed by AI agents that interact with real systems and data, providing a structured view of how agents can be subverted and expanding both the defensive playbook and the community’s collective knowledge base.

Continuing research and community collaboration

Zenity continues to expand research efforts on the AI Agents Attack Matrix and will continue to add new TTPs as AI Agents continue to evolve. The framework is open source, and we invite the community to collaborate, contribute case studies, and share mitigations. By working together, we can ensure the matrix remains current and actionable.

We also encourage researchers and defenders to contribute to MITRE ATLAS, which is a valuable knowledge base for adversary tactics and techniques against AI. Together, these efforts help organizations anticipate and defend against the rapidly developing threat landscape introduced by the use of AI and AI agents.