Prompt Engineering Techniques Shift as Models Absorb CoT

A catalog that keeps growing while the core gets smaller

Prompt engineering keeps producing new technique names. The core practices that hold up in production are getting fewer. Columnist Lance Eliot's A-to-Z guide to prompting techniques, published in June, numbers its checklist past 110 entries. They run from Add-On Prompting through Year-In-Review Prompting, with stops at Atom-of-Thoughts, Chain-of-Verification, Hermeneutic Prompting and Verbalized Sampling along the way.11 The list covers serious reasoning methods, user-safety work such as "cognitive cognizance" prompting, and adversarial material, including a note that attackers have gotten around AI safeguards by phrasing requests as poetry.11

Practitioner guides published this year tell a different story. Their lists stay short and repeat each other: zero-shot, few-shot, chain-of-thought, self-consistency, role prompting, meta-prompting and, for agent workloads, ReAct.121813 One enterprise-focused guide calls zero-shot, few-shot, chain-of-thought, role-based prompting and ReAct the top five techniques for production LLM applications.18

The two views fit together once you see who each one is for. Eliot's catalog maps what people have tried. The shorter lists describe what engineering teams actually ship. The most important change in 2026 is that chain-of-thought, the technique at the center of both, has lost much of its value.

Chain-of-thought: from default to judgment call

Eliot describes chain-of-thought prompting as one of the most celebrated techniques, and several entries on his list are built on it. Atom-of-Thoughts, for example, breaks reasoning into atomic steps that a suitably configured system can run in parallel.11 Most of the recent evidence argues against using these scaffolds by default.

The most cited data comes from the Wharton Generative AI Labs report by Lennart Meincke, Ethan Mollick, Lilach Mollick and Dan Shapiro. On non-reasoning models, asking for step-by-step reasoning raised average scores. Gemini Flash 2.0 gained 13.5% and Sonnet 3.5 gained 11.7%, while GPT-4o-mini's 4.4% gain was not statistically significant.1 Average gains hid a reliability cost, though. On the strictest measure, answering correctly every time, several models got worse; Gemini Pro 1.5 fell 17.2%.1 The authors' reading is that chain-of-thought helps on hard questions but adds enough variability to cause mistakes on easy questions the model would otherwise get right.7

On reasoning models the case is weaker still. o3-mini and o4-mini improved by only about 3%, Gemini Flash 2.5 slipped 3.3%, and chain-of-thought requests took 20–80% longer to complete.1 The researchers concluded that for dedicated reasoning models, the extra benefit of explicit chain-of-thought looks negligible and may not be worth the added processing time.10

Vendor guidance points the same way. One analysis of OpenAI's documentation says the company considers telling its reasoning models to "think step by step" unnecessary, and its o3/o4-mini guidance warns that asking such models to reason more may hurt performance.2 A guide to prompting current frontier models gives two reasons. The model may repeat its internal reasoning in the visible answer, which raises output cost without improving quality, or it may restrict its own exploration to fit the procedure the user wrote out.6 The practical advice is to control reasoning depth through API parameters such as reasoning-effort settings and thinking budgets, not through wording in the prompt.6

Not everyone agrees. One engineering firm still argues that chain-of-thought's 2–3x token overhead is worth paying for reasoning-heavy work such as code generation and financial modeling. It also warns that self-consistency and tree-of-thought can multiply output tokens 5–50x, which at production volume can add six figures a month in cost.4 One 2026 comparison table cites a 19-point gain on MMLU-Pro for standard models, but it tells readers to skip manual chain-of-thought on reasoning models because they already do it internally.14 The most defensible position is the narrow one. Explicit chain-of-thought still helps on older, smaller or local non-reasoning models, and when a team needs a fixed reasoning trail it can audit. Elsewhere it mostly adds cost.2

The effect on Eliot's catalog is direct. Many of its most technical entries are variations on visible stepwise reasoning, including Chain-of-Draft, Doubling-Up Chain-of-Thought, Least-to-Most and Skeleton-of-Thought.11 One practitioner's survey argues that five of these familiar methods actively degrade reasoning models: chain-of-thought, few-shot, self-consistency, least-to-most and skeleton-of-thought.9 That is a strong claim. Still, it suggests a large part of the A-to-Z list was designed for model generations that are now being replaced.

Structured output has moved out of the prompt

Structured output has seen the biggest change in practice. Older guides treated JSON reliability as a prompt-writing problem: state the schema, show one ideal example, list strict formatting rules, and tell the model to check its own work.24 That advice is still useful, but the industry now treats it as a fallback.

One 2026 guide estimates that prompt-only JSON succeeds roughly 80–95% of the time under ideal conditions. At 10,000 records a day, a 5% failure rate means 500 broken extractions.25 Its alternative is native schema enforcement. In OpenAI's strict mode, constrained decoding prevents the model from generating tokens that would break the schema, and the older JSON mode is treated as legacy for any schema-bound work.25 Other guides agree on a key distinction: JSON mode guarantees valid syntax but not a correct schema, while Structured Outputs are built to match a schema.2723 For Anthropic's Claude, one guide recommends defining the schema as a tool and forcing the model to call it.27

Guides also agree that validation stays in place even with native enforcement. They recommend a pipeline of prompt, generate, validate, repair and parse, with Pydantic or Zod models as the single source of truth for both the schema and the validator.2629 One guide puts the division of labor this way: the prompt sets intent, the schema sets structure, and validation sets trust.30 It also warns that output can match the schema and still fail the task, for example by returning empty summaries. That calls for business-rule checks on top of schema checks.30

A more contrarian argument says JSON belongs at application boundaries and in output schemas, but often not in the body of the prompt. Large arrays of repeated JSON objects make the model spend tokens reading serialization syntax instead of data, so compact tables or lossless aliases are better inputs.23

One point needs settling. A guide to structured output recommends adding "think step by step" to JSON prompts,29 which sits awkwardly next to the evidence above. A better pattern, when reasoning is wanted, is to put a reasoning field inside the schema before the answer fields, or to split the job into one free-form reasoning call and a second extraction call.27

Few-shot examples: the context decides

Few-shot prompting shows the conflicting advice most clearly. One structured-output guide says that giving a schema description without examples causes 20–30% failure rates.22 Another argues that examples can overwhelm or redirect a reasoning model's internal process.9 Both can be right, because they describe different jobs. Examples still work well for pattern-matching, format-following and classification. On open-ended reasoning tasks, they anchor the model on the user's solution path and reduce variety in its answers.6 Put simply, examples help with formatting and can get in the way of reasoning.

System prompts have become engineering artifacts

Advice on system prompts has moved toward basic rules and defensive design. Microsoft's Foundry documentation recommends stating the task near the start, grounding the model in supplied data, repeating key instructions where needed, and giving the model an "out", such as answering "not found" when the information is missing.16 DigitalOcean recommends wrapping user content in delimiters or XML tags and labeling it as data rather than commands. It also advises screening inputs for injection phrases, treating model output as untrusted before acting on it, and red-teaming prompts.13 Model-specific styles still matter: one comparison says Claude benefits from XML tags and GPT models from Markdown structure.14 One guide also advises putting binding requirements at the end of the prompt, so they are still prominent when a reasoning model finishes thinking.6

Role prompting, one of the oldest patterns, is losing ground in the same way. Some guides still recommend it for nearly every task.17 More careful comparisons limit it to tone and open-ended work and call it cargo-cult practice for classification and factual Q&A.14 Even supporters warn that elaborate personas add noise.12

What it adds up to

The common thread is a shift from tricks to infrastructure. Enterprise guides now tell teams to version prompts, diff them, roll them back, and test every change against evaluations, because small wording edits that pass obvious cases can break edge cases.18 Others recommend versioning the prompt, schema, validator and test set together.30

Eliot's catalog is still a useful reference for what is possible, including its entries on safety and adversarial prompting. For teams shipping products, the 2026 approach is narrower. Pick a reasoning model where the task needs it and set its effort level through the API. Enforce output structure with schemas instead of instructions. Use examples for format, not for reasoning. Treat every prompt as code that needs tests. Much of the lettered list now describes problems that model providers have built into the models and APIs.