Any model · diagnostic workflow
How to debug a bad AI response
A step-by-step method for reconstructing intent, freezing requirements, assigning the right cause, repairing the request, and judging the retry.
Short answer
What is probably happening
Debug the interaction like a failed system, not like a disappointing conversation. Preserve the original instruction and output, reconstruct the intended result, separate explicit requirements from assumptions, test the output requirement by requirement, assign the primary cause, then change only the part of the process that caused the miss.
Possible causes
The same symptom can have different verdicts
The most common mistake is jumping straight to a rewritten instruction. That can hide whether the model ignored something clear, whether the task lacked information, or whether the first result was already correct against the stated target.
Target reconstruction
You cannot judge a result until the intended artifact and its success criteria are explicit.
Evidence comparison
The original instruction and output must be compared line by line against frozen requirements, not summarized from memory.
Capability and access
The chosen model or tool may lack the source, repository access, exact editing mechanism, or context needed for the job.
Retry control
A retry is useful only when it keeps the target stable and tests one repaired cause.
How to distinguish them
Check the evidence in this order
- 01
Save the exact original instruction, model/version, output, and any attachment.
- 02
Write one sentence describing the intended result without explaining who failed.
- 03
List explicit, confirmed, and inferred requirements separately.
- 04
Mark each requirement met, partial, or missed using the output itself as evidence.
- 05
Choose among instruction, model, both, model choice, task, or no problem.
- 06
Repair only the cause, retry once, and compare against the same requirement list.
WWW-style diagnosis
Example caseyeah… both of you.
- Wanted
- A launch plan grounded only in real meeting decisions.
- Asked
- Turn these notes into a launch plan and fill in the gaps.
- Result
- A polished plan with invented owners, dates, and budgets presented as facts.
- Why
- The instruction invited unbounded gap-filling, while the model also failed to label assumptions. The repair must constrain evidence and make uncertainty visible.
What to do next
Fix the diagnosed cause
If the instruction caused it
Add the missing boundary or success criterion without rewriting the user’s real intent into a different task.
If the model caused it
Retry the clear instruction once, then change model or workflow if the same explicit failure repeats.
If the task or tool caused it
Supply the missing source, narrow the task, grant the required access, or use a mechanism built for the operation.
If nothing went wrong
Keep the result and record the newly discovered preference for the next request instead of blaming the previous output.
When the instruction needs repair
A boundary-first template
Goal: [intended result]. Original instruction: [exact text]. Model and version: [model]. Result: [exact output or relevant excerpt]. Reaction: [specific mismatch]. First reconstruct the intended outcome and list explicit versus inferred requirements. Then classify the primary cause as instruction, model, both, model choice, task, or no problem. Cite the evidence, propose the smallest repair, and preserve the same requirements for the retry.
This template is relevant only when the evidence points to missing or conflicting boundaries. It cannot excuse a model that violated a clear instruction.
Questions
What people usually ask next
Should I immediately rewrite the prompt?+
No. First determine whether missing wording caused the failure. Rewriting a clear instruction can conceal a model or tool problem.
How many retries are useful?+
Usually one controlled retry is enough to test a repaired cause. Repeating the same failure without learning is not debugging.
What should stay fixed between retries?+
The intended outcome and requirement list. Change the suspected cause—wording, model, access, or method—without moving the target.