How many retries should I allow?

Two or three, then fail into a defined fallback. The first retry with well-formatted error feedback fixes the majority of fixable failures; the second catches most of the rest; beyond that you are mostly paying for repetitions of the same mistake, because the failed attempts accumulate in context and can anchor the model on its own error. A hard cap also bounds worst-case latency and cost, which matters once the loop is inside a user-facing request. Log every retry with its validation errors; a rising retry rate is an early warning that a prompt, schema, or model change regressed something.

Should I send the raw validation traceback back to the model?

No. Raw pydantic tracebacks and Zod issue dumps are written for developers and bury the signal in noise. Translate each issue into an instruction: instead of a ValidationError block, send "Your previous response had 2 problems. 1) items[2].qty was the string 'three'; it must be an integer. 2) The required field total was missing. Return the corrected JSON object only." Include the model's previous output so it can edit rather than regenerate. Both Zod and pydantic expose structured error lists (error.issues, e.errors()), which makes this translation a ten-line formatting function.

If schema mode guarantees structure, why keep the retry loop at all?

Because structure is the smaller half of validation. Schema mode, as of 2026-07, makes parse errors and shape mismatches genuinely rare on OpenAI, Gemini, and grammar-constrained local models. But no schema can know that product SKU-99231 is not in your catalog, that a meeting cannot end before it starts, or that a cited quote does not appear in the source document. Those semantic checks live in your code, they fail at meaningful rates on real inputs, and re-prompting with the specific violation is exactly the same loop. Keep the loop; let schema mode empty out its structural half.

Retry Loops for LLM Validation Failures That Converge

Every structured-output pipeline needs an answer to the question "and what happens when the output is wrong?" The mature answer is a small, bounded loop: validate, feed the errors back as instructions, try again, and give up quickly. It is unglamorous, it is about thirty lines of code, and it is the difference between a demo and a system.

This article covers that loop in detail. For the map of how to get structured output in the first place — schema mode, JSON mode, tool-based extraction — see the cluster pillar, how to get reliable JSON out of LLMs.

The pattern

attempt = model(prompt)
for retry in 1..MAX_RETRIES:          # MAX_RETRIES = 2 or 3
    errors = validate(attempt)        # Zod / pydantic / JSON Schema + your checks
    if not errors:
        return attempt
    feedback = format_errors(errors)  # instructions, not tracebacks
    attempt = model(prompt, previous=attempt, feedback=feedback)
return fallback()                     # defined behavior, not an exception leak

Three decisions hide in those lines: what validate checks, what format_errors says, and what MAX_RETRIES and fallback are. Each one has a right answer.

What to validate, in two layers

Layer 1: structural. Does it parse, are the fields present, are the types right? In TypeScript this is Zod (schema.safeParse(data) returns an issues list instead of throwing); in Python it is pydantic (Model.model_validate(data), catching ValidationError and reading e.errors()); language-agnostic stacks use a JSON Schema validator directly. All three give you structured error lists, which matters for the feedback step.

Here is the thing about layer 1 in mid-2026: if you are using provider schema enforcement — OpenAI's json_schema with strict: true, Gemini's responseSchema, a forced Claude tool schema, a llama.cpp grammar — layer-1 failures become rare, though the exact guarantees vary by vendor and truncation or refusals can still produce one, so the check stays. Constrained decoding makes structural violations rare enough to page on rather than retry on. The comparison of those modes is in json mode vs schema mode. Keep the layer anyway: it is free, it catches provider regressions and proxy weirdness, and not every model in your stack will have schema mode forever.

Layer 2: semantic. This is where the loop earns its keep, because no schema can express it. The product ID must exist in your catalog. The end_date must be after the start_date. The quoted sentence must actually appear in the source document. The category must make sense for the department that will receive the ticket. These checks fail at real rates on real inputs — a few percent is typical for extraction tasks — and they are entirely your responsibility. "Valid JSON" and "correct answer" are different claims; schema mode gets you the first, layer 2 defends the second.

Format errors as instructions, not tracebacks

The single highest-leverage detail in the whole pattern. When validation fails, the temptation is to dump whatever your validator produced into the next prompt. Resist it. A raw pydantic traceback is optimized for a developer with a stack frame open, not for a model deciding which token comes next.

Bad feedback:

ValidationError: 2 validation errors for Invoice
items.2.qty
  Input should be a valid integer [type=int_parsing, input_value='three', ...]

Good feedback:

Your previous response had 2 problems:
1. items[2].qty was "three". It must be an integer, e.g. 3.
2. product_id "SKU-99231" does not exist in the catalog. Use only IDs
   that appear in the CATALOG section of the input.
Return the corrected JSON object and nothing else.

The translation is mechanical — iterate over error.issues or e.errors(), render path, problem, and expected form — and it pays off immediately in first-retry success rate. Also include the model's previous output in the retry context, so it can make a targeted edit instead of regenerating from scratch and introducing new errors elsewhere. For semantic failures, say what would have been valid ("use only IDs from the CATALOG section"), not just what was wrong; the model cannot query your database, so the feedback has to carry the constraint.

Cap it at 2 or 3, and mean it

Retry curves for this pattern bend hard. The first corrective retry fixes most fixable failures; the second catches a meaningful remainder; after that you are mostly watching the model repeat its mistake with growing confidence, because the failed attempts now dominate the context. Three other reasons to keep the cap low:

Latency and cost are multiplicative. A 3-retry loop inside a user-facing request is up to 4 model calls before anyone sees pixels.
Some failures are not retryable. If the document genuinely does not contain the field, no amount of re-prompting produces it — it produces a hallucination shaped like compliance. Distinguish "the model formatted this wrong" (retryable) from "the input cannot satisfy the schema" (not retryable; the schema needs a nullable field or a not_found value instead).
Retry rate is a metric. Log every retry with its error list. A stable pipeline retries on a low single-digit percentage of requests; when a model version, prompt edit, or schema change moves that number, you want the alert before your users notice.

The fallback after the cap should be a decision, not an exception: queue for human review, return a partial result flagged as unverified, or fail the request cleanly. In agentic settings the same budget discipline applies to tool loops, which is covered in function calling patterns that work.

If the structured output drives side effects — creating records, sending messages, charging cards — run the full validation loop before the acting step, never after. This is the extract-then-act split: the model proposes, validation gates, deterministic code executes. It also means your retry loop is always operating on a proposal, so a retry is cheap; retrying an action is not. Well-designed tool descriptions reduce how often you enter the loop at all, by making the model's first attempt land more often.

The takeaway

Validate everything in two layers, let provider schema mode absorb the structural layer, and keep a 2-to-3-retry loop for the semantic layer with errors rewritten as concrete instructions next to the previous attempt. Cap it, log it, and route cap-outs to a defined fallback. As of 2026-07 this combination — schema enforcement plus a small semantic retry loop — is the standard reliability stack for structured LLM output, and neither half is optional on its own.

Retry Loops for LLM Validation Failures That Converge

Article summary

The pattern

What to validate, in two layers

Format errors as instructions, not tracebacks

Cap it at 2 or 3, and mean it

One refinement: validate before you act

The takeaway

Frequently asked questions

See also

Where to go next

Comments 0