Welcome to the second part of our series examining how to frame the cyber threat environment in relation to artificial intelligence. As we noted in our first piece, there is a lot of FUD (fear, uncertainly, and doubt) out there around AI and what it means for cyber threats. Some of these concerns are legitimate, but most are not. In fairness, it’s easy to freak out about AI and just assume everything needs to be protected with every tool on the market… but as the saying goes, when everything is important, nothing is important. In this second article, we’ll be looking at a useful framework for how to think about threats TO AI, specifically why companies’ usage of AI presents a unique threat model for defenders to consider and a desirable target for threat actors.
Personally, I find the most clear and compelling model to frame how to think about AI threats at a macro level, particularly when it comes to AI agents, comes from Simon Willison in his Lethal Trifecta, which can be found here. Mr. Willison argues there is a “Lethal Trifecta” when using AI, particularly agents. Specifically, these systems have three characteristics that can allow an attacker to steal your data if you do not consider and understand the risks associated with them. These characteristics are:
- Access to your private data: AI tools almost always have access to some sort of personal and/or proprietary data. Their ability to access and manipulate this data is often what provides the intrinsic value for the AI agents.
- Exposure to untrusted content: There are numerous ways text, images, or other messages controlled by a malicious threat actor could be introduced to your AI agent or model. Most people typically associate this with prompt injection attacks, but I think it’s worth considering larger supply chain attacks, including malicious skills or other connections, as part of this.
- Ability to Externally Communicate: Again, this is often part of the value proposition for AI agents, but it also presents an easy exfiltration path for threat actors. This is how your AI tools can ship your data out your front door.
At the core of these threats are the fact that large language models will generally read and execute instructions without concern for the provenance of the instructions themselves, meaning threat actors can embed malicious instructions into a variety of inputs with the goal of tricking AI models into executing their directions (the typical prompt injection example). This is just a great and deceptively simple way to frame thinking about what threats AI and agents can introduce into your environment, particularly when considered concurrently. As Mr. Willison states, these present a potential “lethal trifecta.”
Earlier this year when OpenClaw went viral, followed in rapid succession by a wave of abuse reports in which threat actors were creating malicious skills or agents to introduce into the OpenClaw ecosystem, Palo Alto added a bit more to Mr. Willison’s model. Palo Alto argued, quite effectively, that a fourth characteristic needs to be considered in relation to AI agent threat models. This fourth characteristic is persistent memory. In the linked post, Palo Alto pointed out that persistent memory allowed threat actors to parse out their malicious payloads over a series of inputs, leveraging the long-term nature of agentic AI memory to execute once the full payload had been delivered. As Palo Alto pointed out in the post, “[t]his enables time-shifted prompt injection, memory poisoning, and logic bomb–style activation, where the exploit is created at ingestion but detonates only when the agent’s internal state, goals, or tool availability align.”
From a threat modeling perspective, this means every input into an LLM or AI agent needs to be considered against the entire corpus of entries. Attacks against AI models aren’t just point-in-time issues … a single attack could in theory exist across the entire timeline of your usage until the point of execution. This means constant monitoring and proactive defense is critical.From a threat perspective, persistent memory could also include the accretion of data and proprietary and/or personal information in AI agents over time as their use increases, which also provides an attractive target for threat actors in and of itself.
Source: Palo Alto Networks
These four characteristics of AI models, particularly AI agents, provide an excellent high-level framework for how to consider your threat environment as you introduce more advanced AI capabilities. This isn’t to scare people or organizations away from using an impressive technology, but rather to allow them to frame the threats in the right way to allow them to make informed decisions on their mitigations and defenses.
In our next post, we’ll be digging in a bit deeper on examining the risks associated with AI and what is within your control depending on whether you are a model consumer or creator using Google’s Secure AI Framework.
