What AI Agents Know About Your Security Holes

OpenAI's crawlers indexed a RubyGems caching vulnerability before it was widely disclosed. Aaron Patterson (tenderlove) documented it. The implication is uncomfortable: AI systems trained on scraped web data may carry knowledge of security flaws your team assumes are obscure.

For anyone building AI agents that touch production systems, this is worth sitting with.

What actually happened

Patterson found that OpenAI's bots had crawled content describing a caching vulnerability in RubyGems. The models trained on that data therefore "know" about it — not because a researcher fed it to them deliberately, but because the web was scraped and the information was there.

This is not a story about OpenAI doing something malicious. It is a story about how LLM training works: broad corpus, broad capture. Security advisories, GitHub issues, mailing list threads, obscure blog posts — all of it goes in. The model does not distinguish between public knowledge and sensitive disclosure.

Why this matters for AI agent builders

AI agents are not static. They query LLMs at runtime, and those LLMs carry latent knowledge about your stack, your dependencies, and their known weaknesses.

If you are building an agent that helps developers write code, reviews PRs, or answers questions about your infrastructure, you need to think about what the underlying model already knows — and what a bad actor could extract from it with a well-crafted prompt.

The attack surface is not just your agent's tool calls and API keys. It is the model's training data acting as an unintentional knowledge base about vulnerabilities in every library your agent might touch.

Concretely: an agent wired to a Ruby codebase, given enough prompt latitude, could surface a known CVE it was never explicitly told about. The model learned it from a GitHub issue three years ago.

The knowledge cutoff is not a safety guarantee

A common assumption: "the model's training data is old, so it does not know about recent vulnerabilities." That assumption has two problems.

First, training cutoffs are longer than people think. A vulnerability disclosed 18 months ago is well within the window for most frontier models.

Second, "obscure" does not mean "uncrawled." Patterson's post illustrates exactly this. RubyGems is not a niche toy — it is core Ruby infrastructure. But even genuinely niche packages get discussed in Stack Overflow threads, dependency audit reports, and CI logs that end up indexed.

The model may know more about your specific stack than you expect.

What to actually do about it

This is not a reason to stop building agents. It is a reason to build them with tighter scope.

Constrain what the agent can see. If an agent does not need access to your full dependency tree or infrastructure config, do not give it that context. Narrow system prompts reduce the surface for both accidental disclosure and prompt injection.

Treat the LLM as an untrusted third party. Any data the model generates about your systems should be validated before acting on it — same as you would treat output from an external API you do not control.

Audit what your agents output, not just what they do. Tool calls and API writes are the obvious risk surface. But an agent that surfaces vulnerability information in a chat interface is also a risk, especially if that interface is customer-facing.

Stay current on your dependencies anyway. An AI agent that knows about a CVE in a library you are running is a lagging indicator that you should have patched it. Automated dependency scanning (Dependabot, Renovate, Snyk) is table stakes before you wire an LLM anywhere near your codebase.

If you want a fast read on where agents could genuinely help your team — versus where they add risk — our free AI Opportunity Audit maps your three highest-impact automations from just your website. Takes a few minutes.

The broader point

LLMs are trained on the public record of how software breaks. Every post-mortem, every CVE writeup, every "I found a weird bug" thread is in there. That makes them surprisingly capable at spotting issues — and it means the knowledge asymmetry you assumed you had over a bot is smaller than it looks.

Build accordingly.

Build with someone who has shipped this

If you are scoping an AI agent for your team and want to think through the security and architecture questions before you start, book a call. No pitch deck — just a practical conversation about what makes sense to build.