Editing Openai/692e337f-58e4-8007-8044-351091f84b8b (section)

==== - There is a well-studied phenomenon called data poisoning: malicious actors deliberately insert corrupted or adversarial data into training or retrieval datasets for machine-learning models. IBM<ref>{{cite web|title=IBM|url=https://www.ibm.com/think/topics/data-poisoning|publisher=ibm.com|access-date=2025-12-01}}</ref> ====
* According to a recent systematic review, there are multiple classes of “poisoning attacks” against large language models (LLMs), including “persistent poisons,” “stealthy poisons,” “concept poisons,” and “task-specific poisons.” arXiv<ref>{{cite web|title=arXiv|url=https://arxiv.org/html/2506.06518v1|publisher=arxiv.org|access-date=2025-12-01}}</ref>
* Empirical research demonstrates that even a very small fraction of malicious documents can be enough to change model behavior. For example, poisoning only 0.1% of pre-training data was sufficient for “belief manipulation, jailbreaking, or backdoor” behavior to persist even after fine-tuning. arXiv<ref>{{cite web|title=arXiv|url=https://arxiv.org/abs/2410.13722|publisher=arxiv.org|access-date=2025-12-01}}</ref>
* Another more recent study found that as little as 250 malicious documents (≈ 0.00016% of total training tokens) sufficed to backdoor models up to 13B parameters — suggesting that model size does not inherently defend against poisoning. Anthropic<ref>{{cite web|title=Anthropic|url=https://www.anthropic.com/research/small-samples-poison|publisher=anthropic.com|access-date=2025-12-01}}</ref>
* Beyond pre-training, “retrieval-augmented generation” (RAG) pipelines are also vulnerable: attackers can poison the retrieval corpus (e.g. web data, document stores) so that when the AI fetches context, it includes manipulated or malicious content — affecting outputs. arXiv<ref>{{cite web|title=arXiv|url=https://arxiv.org/html/2402.07867v1|publisher=arxiv.org|access-date=2025-12-01}}</ref>

Implication: the general concept of “AI poison” — i.e. corrupting training/knowledge data to alter model outputs or embed backdoors — is real, documented, and nontrivial to defend against.