AI Data Poisoning Attacks

A model is a summary of the data it saw. Poisoning changes that data on purpose.

An attacker can reach three places. The training or fine-tuning set. The model file itself, for example an open model downloaded from a public hub. And the retrieval layer: the documents, tickets and wiki pages a retrieval-augmented generation (RAG) system pulls in to answer a question. MITRE ATLAS tracks that last one as its own technique, AML.T0070 RAG Poisoning, separate from AML.T0020 Poison Training Data.

It is quiet, because a poisoned model works normally most of the time, and durable, because retraining on the same source reproduces it.

We use the categories in NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (March 2025), plus RAG poisoning. The right-hand column is our opinion for 30 to 200 person companies, and a poison-detection vendor would dispute it.

The OWASP Top 10 for LLM Applications covers this under LLM04:2025 Data and Model Poisoning and flags "malicious pickling", where a model file runs code when loaded. We treat model downloads like any other third-party package.

Founders usually hear about poisoning through one dramatic statistic. Here is what the evidence found, and what it does not prove.

In October 2025, Anthropic, the UK AI Security Institute and the Alan Turing Institute published a study that trained 72 models from 600M to 13B parameters. About 250 poisoned documents, roughly 0.00016% of training tokens for the largest model, reliably planted a backdoor at every size; 100 did not. The absolute number of documents mattered, not the percentage. The caveat: the backdoor only produced gibberish after a trigger phrase, and the authors say it is unclear whether this holds for harmful behaviors or larger models.

PoisonedRAG, presented at USENIX Security 2025, reported a 90% attack success rate when five malicious texts were injected per target question into a knowledge database holding millions of texts. If outsiders can write to what your assistant reads, a handful of pages can steer a chosen answer.

Carlini and colleagues showed an attacker could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for about $60 by buying expired domains the datasets still pointed to,. "We only trained on public data" is not a defense.

A May 2025 joint information sheet on AI data security from NSA, CISA, FBI and partners in Australia, New Zealand and the UK names the data supply chain, maliciously modified data and data drift as the three risk areas to manage.

These two get confused in nearly every AI risk conversation we run. The test is timing and persistence.

The overlap is RAG. A poisoned page in your knowledge base can carry instructions, which is indirect prompt injection delivered through poisoning; MITRE's RAG Poisoning entry lists prompt injections as one payload. So when a vendor says "we handle prompt injection," ask separately who can write to the content the model retrieves. For the runtime side, see our guide on how to secure AI agents.

You need no ML team for these, just someone who owns the AI feature and checks it weekly.

In our assessments the third item is the one we find most often. It is also the cheapest to fix.

You probably do not need poisoning-specific work if you only use AI through vendor products (a CRM assistant, a coding copilot) and feed them no retrieval content of your own. Your controls are vendor due diligence and contract terms on training data use. The same goes if basic access control is weak: poisoning needs write access, and attackers with write access have easier options. The IBM Cost of a Data Breach 2026 report found 92% of organizations with an AI-related breach lacked proper AI access controls. Fix that first, along with the fundamentals in a cybersecurity baseline assessment.

What this does not stop: nothing below detects a well-crafted backdoor inside a third-party foundation model. You are reducing exposure, not eliminating it.

You are in scope if you run RAG over content customers, partners or the public can write to, fine-tune on user-generated data, or sell an AI feature that makes decisions about people. In Canada, PIPEDA's safeguards principle requires protecting personal information against unauthorized modification, and Quebec's Law 25 adds notice obligations when decisions are made exclusively by automated processing. In the US, HIPAA's integrity standard applies when health data flows through the model.

This is the order of operations only. The full control set and vendor questionnaire are in our data poisoning controls guide.

List every source each AI feature trains on or retrieves from, and who can write to it. Remove broad edit rights. No AI inventory yet? Start with shadow AI discovery.

Version datasets and retrieval indexes so you can roll back. Write 20 to 50 "golden questions" with known answers and run them after every import or retraining. Record the source and hash of every downloaded model. Our MLOps pipeline security guide covers the build side.

Name one owner for AI data integrity, add a poisoning scenario to your incident response plan, and decide whether you need a formal management system such as ISO 42001. Many clients run this through a Virtual CISO engagement; the Crawling tier starts from $2,000 a month for 15 to 20 hours.

Yes. With RAG, the target is your knowledge base, not the model. With downloaded open models, the file itself can be modified. You only escape it if you use a vendor product with no data of your own feeding it.

Usually through behavior: specific wrong answers, unexplained changes after imports, or odd responses to unusual inputs. Versioned sources let you compare against a known-good state and roll back.

The Act names data poisoning among the attacks high-risk AI systems must resist, but those obligations now apply from 2 December 2027 or 2 August 2028 depending on the system. Our AI governance services help Canadian and US SaaS teams plan for it, and you can book a consultation to scope your exposure.