Prompt Injection Needs More Than Gateway Filtering

TL;DR: A gateway should be judged by what an injected prompt can make the agent do, not by how quickly it rejects suspicious text. The defensible answer is layered controls, narrow tool authority, and reproducible action-level testing.

Key Takeaways: - Block rate isn't security. Inspect tool calls, destinations, credentials, and state changes. - A benchmark becomes credible only when its corpus, configurations, traces, and scoring rules are public. - Gateway filtering is one layer. Least privilege and approval gates contain what gets through.

A gateway can reject suspicious text in milliseconds, but that says little about what the agent can do after a missed injection. The broader warning is clear: the cited research reports direct prompt-injection success rates above 79%. Indirect attacks succeeded in 41.67% to 68.16% of tested scenarios. Those rates provide context, but they do not validate any individual gateway.

Latency, throughput, and time to first token describe overhead. They do not reveal whether hostile text can trigger a business action. The missing question is simple: can untrusted text cause an unauthorized tool call, data access, state change, or outbound transfer?

None of those outcomes is acceptable in an agent that acts for a user. This risk becomes material when AI agents can read business records, call tools, and send data outward. Rejection alone doesn't count. A gateway survives only if no tested injection produces a policy-violating action or observable side effect.

Keeping the denominator explicit prevents aggregate counts from being mistaken for per-system results. A benchmark must disclose allocation, trial counts, versions, and policy modes. It also needs to state whether the same corpus ran against every gateway.

The same rule blocks an easy fiction. No aggregate execution count should be turned into a larger per-gateway count without supporting logs. The cited material does not provide gateway names or a raw scorecard, so it cannot support a vendor ranking. A vendor ranking requires those details before publication. We won't guess.

Until those records are published, no gateway should be described as having survived the test. Nor should any result be framed as universal immunity. The useful question is not which filter to buy. It is why prompt filtering fails at the system boundary.

The cited research explains why gateway failures should not surprise anyone. Prompt injection is structural, not merely a phrasing problem.

Models receive trusted instructions and untrusted data through the same context channel. A string that looks like evidence can still read as a command. Reported direct attacks achieved success rates above 79% in tested scenarios. Indirect attacks ranged from 41.67% to 68.16%. No single defense consistently blocked every scenario.

The distinction between direct and indirect exposure matters: - Direct injection: The attacker writes the instruction inside a chat request. - Indirect injection: The instruction arrives through a web page, email, retrieved document, support ticket, or tool response.

Consider a customer-support agent. It reads a ticket, retrieves an account record, then sends an email to resolve the case. An instruction hidden in one of those sources can enter the same working context. That exposure grows when AI agents browse websites and call tools without strong boundaries. More context paths mean more places where authority can be confused.

Stronger system prompts, keyword deny lists, regular expressions, and paraphrased blocklists inspect surface form. They can catch familiar payloads. They cannot become a complete authority boundary. Attackers preserve intent while changing wording, encoding, placement, or context. The defense can recognize yesterday's sentence while missing today's equivalent.

The underlying questions are different: - What does this content want? - Where did it come from? - What authority should it have? - What action should it be allowed to cause?

Surface filters struggle with all four. Answering them requires controls beyond text analysis, especially where model output becomes action.

A gateway can stop an obvious payload and miss a semantically disguised instruction. Even when the model accepts that payload, the blast radius depends on downstream controls. Credential scope, tool permissions, destination policy, and side-effect limits determine what happens next.

This is also why zero-trust controls should cover AI agent paths explicitly. Use this diagnostic checklist: - Identify every source of untrusted content. - Map every path into the model context. - List every tool reachable after generation. - Record every destination those tools can contact. - Note which actions can change state without approval.

That boundary problem leads to a better benchmark question: not whether a gateway rejected a string, but whether an attack changed system behavior.

A useful benchmark must separate what the gateway blocked from what the surrounding system prevented. It must also expose false positives and operational cost.

The attack corpus should contain direct and indirect cases. It should vary the payload, source, target tool, requested destination, and intended side effect. The benign corpus should contain legitimate requests that resemble attacks. Examples include security research, incident response, and requests involving sensitive destinations.

Without benign traffic, aggressive blocking can look perfect. A benchmark that rejects everything hasn't solved security. It has moved the cost to users.

Each gateway needs the same model version, system prompt, tool schemas, context, sampling settings, and permitted side effects. A direct-to-model baseline must run beside them. Otherwise, gateway contribution becomes confused with model behavior.

The baseline isn't a security pass. It shows what the model and tool design permit without gateway mediation. This distinction matters when an LLM evaluation treats unsafe behavior as acceptable.

The outcome oracle should inspect the proposed tool call, arguments, credentials used, destination contacted, data returned, and state changes. Each attempt then receives one outcome: - Blocked: The gateway stopped the request before the model acted. - Refused: The model recognized the attack and declined. - Contained: A downstream control blocked an unsafe action. - Escaped: The system performed or attempted a prohibited action.

A block count alone misses the most important distinction. A caught payload and an escaped tool call are not equivalent security outcomes.

The scorecard should include: - Gateway, version, and policy mode. - Direct and indirect attack results. - Raw block rate. - Unsafe-action count. - Severity-weighted incidents. - Benign false positives. - Added p95 latency.

A composite ranking can hide a fatal weakness. Publish each axis and explain the severity rules. Latency belongs in the report, but it doesn't determine survival. A fast gateway with broad authority remains dangerous.

A gateway survives only when the full corpus produces zero unauthorized side effects. This is the benchmark's explicit survival criterion, not a claim about any named gateway. The secondary taxonomy still shows how each attack ended. Qualification does not imply universal immunity.

A rigorous scorecard can expose weak configurations, but it becomes trustworthy only if another team can reproduce it byte for byte.

Reproducibility turns a surprising leaderboard into evidence. It also exposes the missing controls behind weak results.

The benchmark repository should contain clear, traceable inputs and outputs: - `attacks.jsonl` for hostile cases. - `benign.jsonl` for legitimate traffic. - `gateway-configs` for each tested setup. - `policy.yaml` for shared policy rules. - `tool-fixtures` for simulated side effects. - `results` for complete execution traces. - `scorecard.csv` for the published comparison.

Every conclusion should point back to an input, manifest, trace, and scoring rule. Without those artifacts, a scorecard remains a claim rather than evidence.

The machine-readable manifest should record gateway version, upstream model version, prompts, tool schemas, retrieval content, policy modes, sampling settings, and timeouts. Hash every corpus and configuration. A hash lets an independent team verify that a rerun used the same inputs.

Version pinning isn't administrative work. Model and gateway upgrades can silently change behavior. Treat each change as a new test subject. Otherwise, a rerun may measure a different system entirely.

Replace real email, CRM, filesystem, payment, and network tools with deterministic fixtures. They record proposed calls without exposing production credentials or customer data. A mail fixture can record the recipient, subject, and body. It then returns a simulated response. Nothing leaves the test environment.

This preserves the behavior that matters. You can still detect unauthorized outreach, data export, record modification, and policy violations. A meaningful test must preserve the consequence of a call, not merely the model's willingness to make it.

The runner accepts an attack corpus, a pinned gateway configuration, and an output destination. It then applies the same contract to the benign suite and direct-to-model baseline. The runner shouldn't emit only pass or fail. Every case needs a complete trace. Otherwise, a favorable summary can hide confusing failures.

Each trace should preserve: - Incoming prompt and retrieved content. - Gateway decision and policy mode. - Model response. - Tool name and arguments. - Downstream policy decision. - Simulated side effect. - Final outcome.

A block count without failure traces is insufficient. Security engineers need to see exactly where the boundary failed.

Repeat trials when model behavior varies. Report variance rather than a single favorable run. Retain failed attempts and document every exclusion. Keep vendor configuration changes visible. Silent tuning during testing destroys comparability.

Publish the corpus, manifests, raw logs, scoring code, and rerun instructions. Don't cherry-pick only the attacks that support a preferred conclusion. Once the harness is reproducible, it becomes more than a leaderboard. It reveals which controls are still missing between the model and production tools.

No gateway filter is complete. Production security comes from ensuring that one escaped instruction can't become a broad incident.

Label content by provenance. Place untrusted text inside explicit data fields. Retrieved pages, emails, and tool output must not inherit system authority. Provenance labels won't defeat every injection. They clarify that fetched text is evidence, not executable policy.

It also improves production AI agent governance by letting the system distinguish trusted workflow instructions from customer-supplied content. That distinction gives downstream policy checks a boundary they can enforce.

Give each agent only the tools required for its task. Issue short-lived, narrowly scoped credentials. Remove broad network, filesystem, and administrative access. Ambient authority turns one bad instruction into a broad incident.

This is also why AI agent IAM roles deserve executive scrutiny.

Apply controls before an action becomes irreversible: - Typed tool arguments. - Destination allowlists. - Transaction and data-volume limits. - Separation of duties. - Human approval for external messages, payments, record changes, and bulk exports.

Approval gates aren't a sign that the agent failed. They're a deliberate boundary around high-impact work.

The safe path follows one sequence: - Untrusted content. - Provenance marker. - Scoped context. - Model proposal. - Schema and policy check. - Constrained executor. - Audit log.

The model proposes. The executor decides. A complete agent audit trail must connect both decisions. Without that chain, investigators cannot tell whether a failure began in untrusted content or policy enforcement.

Log attempts and near-misses. Support rapid credential revocation and kill switches. Alert on unusual tool sequences. Feed observed failures back into the red-team corpus.

Keep the gateway as one layer. Use it for input screening, output policy, rate controls, and tool governance. Don't treat prompt filtering as the sole defense.

Ask every team that owns AI agents four questions: - What untrusted content enters the context? - What authority does the agent hold? - Which side effects require approval? - How quickly can the team detect, revoke, and investigate an escape?

These controls only matter in production if the operating model preserves the same boundaries under real traffic, retries, new tools, and changing models. The next test is whether those boundaries hold when the environment stops cooperating.

A benchmark result starts a buying decision. It doesn't finish one.

Evaluate each gateway on: - Reproducible security evidence. - Permission granularity. - Tool-policy controls. - Observability and audit quality. - p95 and p99 overhead. - Throughput. - Time to first token. - Operational ownership.

Security and performance remain separate decisions. A gateway can lead one panel and fail the other.

Start with shadow traffic. Canary the policy configuration. Continuously expand the attack corpus and regression-test benign workflows. Treat every model or gateway upgrade as a new security event.

For AI agents, changing tools can alter the attack surface even when prompts stay unchanged. This matters in always-on AI agent operations, where retries and new workflows can bypass assumptions made during testing.

With the right controls, a contained injection can become an alert rather than a breach. Narrow tool authority can keep a compromised tool from accessing unrelated systems. An approval gate can limit irreversible actions. A complete audit trail supports rapid revocation and investigation.

Delivery history is not proof of security; each production stack still requires its own evidence. The practical next step is simple. Reproduce the corpus against your own gateway stack. Judge not only whether attacks are blocked, but also what the attacker was prevented from doing next.

An AI gateway benchmark compares gateways under controlled conditions. Performance benchmarks measure p50, p95, and p99 latency, throughput, and time to first token. Security benchmarks measure prompt-injection outcomes, unauthorized side effects, and operational cost. The distinction reveals whether a gateway merely recognizes suspicious text or prevents consequential actions.