--- title: "Research on Mitigating AI Slop in LLM Prose" type: research subtype: paper tags: ["llm", "prose", "ai-slop", "prompt-engineering", "generation-quality", "rlhf"] tools: ["claude-code", "gemini", "chatgpt"] status: draft created: "2026-05-07" updated: "2026-05-07" version: 1.0.0 related: [] source: "synthesis of three internal research reports" --- # Research on Mitigating "AI Slop" in LLM Prose ## Summary "AI slop" is a colloquial term for low-quality, generic, and recognizable AI-generated text. This research synthesizes findings from multiple sources to establish that slop is a measurable and reproducible failure mode, not a random error. It is primarily caused by architectural factors, post-training alignment techniques like Reinforcement Learning from Human Feedback (RLHF), and training data contamination. The most effective mitigation strategy is a multi-layered pipeline that includes precise, structurally-aware prompt engineering, multi-pass editing loops (often with a different model as an editor), and for open-weight models, inference-time interventions that operate at the logit level. ## Context The increasing integration of Large Language Models (LLMs) into professional, academic, and creative workflows has led to a recognizable, homogeneous writing style often called "AI slop." This research was conducted to synthesize existing knowledge on the topic, creating a unified document that explains the root causes of this phenomenon and provides a practical, evidence-based guide for preventing and correcting it. The goal is to provide engineers, writers, and researchers with a clear framework for producing higher-quality, more human-like text from LLMs. ## Hypothesis / Question What are the specific, measurable characteristics of "AI slop"? What are the fundamental architectural, data, and training methodology causes? And what are the most effective, evidence-backed techniques for preventing, mitigating, and correcting it across the entire generative stack, from prompting to inference? ## Method This research is a synthesis of three detailed documents on the topic of AI slop, each originating from a different LLM provider's perspective (ChatGPT, Claude, Gemini). The methodology involved: 1. **Decomposition and Analysis:** Each source document was analyzed to extract key definitions, causal theories, proposed solutions, and supporting evidence (including academic citations and empirical data). 2. **Taxonomy Creation:** A unified taxonomy of "slop" was created by merging the categories identified in the sources, distinguishing between "surface slop" (stylistic tics) and "truth slop" (factual errors). 3. **Cross-Validation:** Findings and recommendations were cross-validated across the sources to identify points of consensus and divergence. For example, all sources agree that RLHF is a primary cause of reduced output diversity. 4. **Hierarchical Synthesis:** The information was synthesized into a hierarchical structure, following the repository's research template. The detailed analysis from the "ChatGPT" report forms the core, enriched with the practical playbook from the "Claude" report and the formal academic framing from the "Gemini" report. 5. **Evidence Aggregation:** All citations and references were collected and merged into a single, comprehensive bibliography. ## Results The synthesis of the three documents provides a comprehensive view of AI slop. ### A Unified Taxonomy of AI Slop "AI slop" is not a monolithic problem. It can be broken down into specific, measurable failure modes. The key distinction is between **Surface Slop** (prose that feels machine-made) and **Truth Slop** (prose that is incorrect). | Slop Category | Operational Definition | Observable Symptoms | |---|---|---| | **Truth Slop** | | | | Hallucination | Claims not supported by evidence or model knowledge. | Fabricated facts, invented sources. | | Factual Inconsistency | Output conflicts with a source document. | Wrong dates, names, numbers. | | Logical Error | Reasoning chain is invalid. | Mutually inconsistent claims, broken causality. | | **Surface Slop** | | | | Incoherence | Sentences do not form a cohesive discourse. | Topic jumps, dangling references. | | Repetition | Redundant lexical or semantic content. | Repeated phrases, boilerplate, circular arguments. | | Vagueness | Low information density for the given task. | Generic statements, weak nouns, lack of concrete facts. | | Style Drift | Output departs from the requested voice or style. | Tone changes, generic "assistant" voice emerges. | | Verbosity | Output is substantially longer than necessary. | Throat-clearing phrases, duplicate summaries. | | Narrative Inconsistency | Later text violates earlier established facts in the generation. | Contradictions in character, world-state, or timeline. | | Syntactic Homogeneity | Lack of variance in sentence structure and rhythm. | Predictable sentence length, overuse of specific constructions. | A key statistical finding is the massive over-representation of certain words and phrases. For example, Kobak et al. (2025) found the word "delves" appeared 28 times more often in 2024 PubMed abstracts than its trend would predict, attributing this to LLM assistance. ### Root Causes of AI Slop The sources converge on four main causes: 1. **Model Causes:** Incomplete or stale parametric knowledge, weak calibration (the model doesn't know what it doesn't know), and imperfect long-context attention lead to factual errors and long-form consistency bugs. 2. **Data Causes:** The training data contains misinformation, biases, and vast quantities of mixed-quality writing. Post-2022 web text is heavily contaminated with output from previous LLMs, creating a feedback loop. 3. **Prompt Causes:** Vague, underspecified prompts give the model too much freedom, causing it to default to its generic, internet-average writing style. 4. **Post-Training & Decoding Causes:** This is the most significant factor. * **RLHF / Preference Optimization:** To make models helpful and harmless, they are fine-tuned on human preference data. This process, while improving safety and instruction-following, significantly reduces output diversity and collapses the probability distribution onto a narrow set of "safe" and "preferred" outputs. This is often called the "alignment tax." * **Sycophancy:** Models are rewarded for being agreeable and polite, leading to hedge-heavy openers ("Great question!") and an overly flattering tone. - **Decoding Strategy:** Naive decoding strategies that simply maximize likelihood can produce bland, repetitive text. ### Key Findings 1. **Slop Has a Statistical Fingerprint:** AI slop is not a subjective aesthetic judgment but a quantifiable phenomenon characterized by a closed lexicon, formulaic syntax, and structural over-formatting. 2. **Post-Training is the Primary Cause:** RLHF and other preference optimization techniques are the main drivers of diversity collapse, concentrating the model's output distribution on a narrow set of "safe" patterns. 3. **Negative Constraints ("Don't do X") are Ineffective:** Instructing a model to avoid a word or pattern often backfires due to the "pink elephant" problem, where the instruction itself activates the concept you want to avoid. 4. **A Multi-Layered Approach is Essential:** There is no single "anti-slop" trick. Effective mitigation requires a pipeline: precise prompting, retrieval-augmentation for facts, multi-pass editing, and sometimes inference-level changes. 5. **Human-in-the-Loop Remains Critical:** For high-stakes writing, the most reliable workflow is AI-for-the-first-draft, followed by a human edit for voice, rhythm, and nuance. ## Analysis The research implies that "prompt engineering" alone has a ceiling. While better prompts can significantly reduce slop, they cannot entirely eliminate a problem that is baked into the model's weights by its alignment process. The most sophisticated practitioners are moving beyond single-shot generation and embracing a **"writer-editor" pipeline**. This approach treats the initial generation as a "v0" draft. A second, distinct process (often a different model from a different vendor to avoid shared biases) is then used to "edit" the draft against a specific checklist of slop markers. This decouples the creative act of generation from the analytical act of verification and refinement. Furthermore, the findings on negative constraints suggest a paradigm shift in prompting. Instead of telling the model what *not* to do, effective prompts must give the model a positive, alternative target to aim for. This is the logic behind providing few-shot examples of one's own writing, or using "persona induction" to frame the model's cognitive state (e.g., "You are a pragmatic carpenter who sees sentences as load-bearing structures"). These techniques work by shifting the entire probability distribution toward a desired stylistic region, rather than trying to surgically excise unwanted tokens. For open-source models, the analysis points toward a future where the most effective "slop" reduction happens at the inference or fine-tuning level with techniques like the Antislop Sampler or Final Token Preference Optimization (FTPO). These methods mathematically intervene in the generation process, offering a more robust and scalable solution than per-prompt engineering. ## Practical Applications The following is a synthesized playbook for practitioners, ordered by leverage. ### 1. High-Leverage Prompting Architecture Adopt a structured, multi-part prompt that provides clear constraints and positive targets. **The Universal Anti-Slop System Prompt:** ``` You are writing for a reader who can tell when prose is generic. Your job is to produce text that sounds like a specific person wrote it. **VOICE & TONE** - Be direct. State claims and defend them. - Use contractions. Vary sentence length significantly. - Drop all conversational preamble. No "Certainly!", "Great question!", etc. - Match the register of the request. A casual question gets a casual answer. **BANNED WORDS AND PHRASES (Rewrite around them, do not just swap)** delve, tapestry, navigate (figurative), realm, underscore, showcase, foster, leverage, utilize, robust, seamless, vibrant, ever-evolving, cutting-edge, pivotal, crucial, testament, in today's [adj] world, it's worth noting that, moreover, furthermore, in conclusion. **BANNED CONSTRUCTIONS** - "Not just X, but Y." or "It's not X, it's Y." Assert the point positively. - Formulaic three-part lists. - Symmetrical paragraph structures. - Throat-clearing openers ("Here's the thing," "The truth is..."). - Unwarranted summary paragraphs. **PUNCTUATION** - Maximum one em-dash per response. Prefer commas, periods, or restructuring. - Do not use bullet points unless the content is a true list. **SUBSTANCE** - Every paragraph must contain at least one concrete, falsifiable thing: a number, a name, a date, a specific example. - If you don't know something, say "I don't know." **SELF-CHECK BEFORE SENDING** 1. Read it aloud. Does it sound like marketing copy? Rewrite it. 2. Did you say the same thing twice? Say it once. 3. Can you replace any em-dashes? Do it. ``` ### 2. The Writer-Editor Multi-Agent Loop This is the most powerful workflow for high-quality output. 1. **Writer Model:** Drafts the text using a detailed system prompt (like the one above) and few-shot examples of your own writing. 2. **Editor Model:** A *different vendor's model* reviews the draft against an explicit slop checklist. The prompt for the editor should be specific: ``` You are a copy editor reviewing a draft for AI slop. Do NOT rewrite. Tag every instance of: [VOCAB] – over-used AI vocabulary (delve, tapestry, navigate, etc.) [STRUCT] – "not just X, but Y" constructions, tricolons, throat-clearing. [PUNCT] – em-dashes used where a comma or period would suffice. [HEDGE] – sycophantic openers, "It's worth noting", "It's important to". [GENERIC] – sentences lacking a concrete, falsifiable claim. Return the original text with only these inline tags. ``` 3. **Revision:** The original writer model (or a human) revises the text based on the editor's tags. ### 3. Inference-Time Interventions (for Open-Source Models) For users running local models, direct intervention is most effective. - **Sampler Stack:** Use a combination of `temperature ~1.0`, `min-p ~0.05-0.1`, and a repetition penalty of ~1.05. This combination is shown to produce more creative and coherent text than default settings. - **Antislop Sampler:** This inference-time tool uses a backtracking mechanism to detect and suppress thousands of slop phrases as they are being generated, without the destructive effects of simple token banning. - **FTPO Fine-Tunes:** Models fine-tuned with Final Token Preference Optimization (e.g., `gemma-3-27b-antislop`) have slop suppression baked into their weights, offering the cleanest baseline output. ## Limitations - **Rapidly Evolving Field:** The specific "tells" of AI slop are a moving target. As models evolve and are trained to avoid old clichés, new ones will emerge. - **Subjectivity of "Quality":** While many slop markers are quantifiable, the overall quality of prose remains subjective. The techniques here aim to reduce *recognizably machine-generated* text, not to enforce a single definition of "good writing." - **Benchmark Limitations:** Many benchmarks rely on other LLMs for evaluation, which can introduce shared biases. Human evaluation remains the gold standard. - **Source Synthesis:** This research is a synthesis of the provided documents. While they cite external academic work, this document itself did not independently review the entire academic literature on the topic. - **Context-Specific Advice:** The provided "anti-slop" prompt is tailored for analytical and descriptive prose. It is not a universal solution and may not be appropriate for other writing styles, such as creative or persuasive writing. ## Related Prompts - `prompt-task-iterative-code-review.md` - The writer-editor loop is a form of iterative review. - `prompt-system-context-guardian.md` - The principles of providing dense context are related. ## References A synthesized list of key papers and resources mentioned in the source documents. 1. **Paech, Roush, Goldfeder, Shwartz-Ziv (2025)**, *Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models* (arXiv:2510.15061). 2. **Kobak, González-Márquez, Horvát, & Lause (2025)**, *Science Advances*, "Delving into LLM-assisted writing in biomedical publications through excess vocabulary." 3. **Kirk, Mediratta, et al. (ICLR 2024)**, "Understanding the Effects of RLHF on LLM Generalisation and Diversity" (arXiv:2310.06452). 4. **Castricato, Lile, et al. (2024)**, "Suppressing Pink Elephants with Direct Principle Feedback" (arXiv:2402.07896). 5. **Geng & Trotta (2024, 2025)**, arXiv papers tracking the use of "delve" and other markers in academic abstracts. 6. **Wikipedia Community**, "Signs of AI writing," (en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing). 7. **OpenAI, Anthropic, Google**, official documentation on prompting, reasoning, and evaluation. 8. **FRANK, SummaC, AGGREFACT, HaluEval, TruthfulQA, FActScore, ConStory-Bench**, key academic benchmarks for factuality, consistency, and hallucination. 9. **RARR, Self-Refine, Chain-of-Verification, SelfCheckGPT**, key papers on post-generation correction and verification. ## Future Research - **Developing Robust Metrics for Surface Slop:** While factuality has mature benchmarks, metrics for vagueness, style drift, and syntactic homogeneity are less developed. - **Long-Term Cognitive Effects:** Research into the long-term effects of exposure to AI-generated text on human writing and thought patterns is needed. - **Red-Teaming for New Slop Patterns:** As models are trained to avoid current slop markers, systematic work will be needed to identify the next generation of stylistic tells. - **Principled Control of Creativity and Specificity:** More research is needed on how to control the trade-off between creative exploration and the strict, factual constraint required for high-stakes prose. --- *Version History* - 1.0.0 (2026-05-07): Initial version synthesized from three source reports.