Why AI 'Guardrails' Aren't Enough

Many AI security programs depend on guardrails to ensure that LLMs and AI agents act in a legal, ethical, and compliant fashion. Guardrails might limit the information that an AI agent can provide, like Anthropic Fable''s restrictions on cybersecurity topics, or otherwise control the content and language of a model''s outputs.

While guardrails are an important part of an AI security strategy, they''re not always effective and certainly not enough for security. Guardrails should be a single layer of a defense in depth strategy that also manages what an AI tool can do, not just say.

What Guardrails Fall Short

Guardrails on AI systems operate at the level of the prompt or output. Their purpose is to restrict what users can ask the model to do or what outputs it can produce.

These guardrails have their limitations and can fall short for several different reasons.

Pretexting

Most LLM chatbots have built-in guardrails to prevent users from asking for illegal or harmful things. However, they''ll generate non-compliant content if asked in the right way.

For example, if you ask an LLM chatbot like Claude to generate a phishing email, it''ll refuse. However, a request framed as writing an email with a benign-sounding purpose can produce phishing content, as shown below.

This screenshot, from this chat, produces what, in any other context, would be called a phishing email. Lower down, the email even suggests calling the IT Help Desk for assistance and provides a number. Ideal for setting up a help desk scam. However, the guardrails failed to trigger, and the LLM wrote the email.

The screenshot above shows that this is even true for emails written using Fable (see chat), which is protected by guardrails that Anthropic itself says are overly restrictive for cyber content. The LLM considers the fact that the email looks and sounds like phishing and writes it anyway, adding only a single, easily-deleted line about typing URLs directly rather than clicking a link.

Anthropic is a frontier AI lab developing one of the most advanced AI tools around. If their guardrails don''t work even when set to be overly restrictive, whose will?

Malicious Outputs

AI guardrails are focused on the prompts sent to an LLM and the outputs that they produce. However, producing incorrect or malicious outputs isn''t the only way that AI can do harm.

With agentic AI, LLMs are built into automated workflows and connected to various tools. They can independently access data, send commands, and influence workflows with little or no human oversight.

A guardrail is designed solely to ensure that an LLM doesn''t produce a malicious output. It doesn''t protect against using tools in a way that hurts the user or the organization.

AI Security Beyond Guardrails

Guardrails can be an important part of an organization''s AI security policy. Preventing LLMs from reading, processing, or leaking PII is essential from a compliance perspective.

However, guardrails are only one element of a multi-layered AI security strategy. Complementary controls include:

• Access and Permission Scoping: AI tools are often assigned privileges on an as-needed basis, and access can accumulate over time. AI privileges should be strictly scoped to the task at hand, and entitlements should be revoked as soon as they are no longer needed.
• Tool-Level Access Controls: Beyond managing overall access to tools, organizations should also manage what an AI system can do with a particular tool. For example, there''s a big difference between the damage that can be done with read-write vs. read-only permissions.
• Ongoing Monitoring: As AI agents become more autonomous, there are more opportunities for them to go off the rails. Continuous monitoring is essential to ensure misunderstandings, hallucinations, or prompt injections don''t result in a rogue agent taking harmful or non-compliant actions.

AI security needs to include more than guardrails and requires a clear understanding of the potential risks and attack vectors. To dive deeper into the example above and other AI threats, check out the Detecting AI-Driven Attacks course on Pluralsight.