Field Notes
A customer writes into support at 8:13 PM.
They are frustrated. They tried setup twice. The instructions missed the screen. They want proof somebody is paying attention.
So an AI drafts the reply.
It is decent. Polite. Clear enough.
Then a person reviews it and says, "No, do not promise that. We changed the policy last week. Also, this customer already tried step three, so stop telling them to try step three again."
Here is the part that matters.
If that correction only lives inside the human's head, the model did not really get better.
The next support ticket comes in, the AI makes the same mistake, and everyone quietly decides AI is inconsistent.
But if that correction gets captured, the system changes.
Policy updates.
The example gets saved.
The support workflow learns that this product issue needs a different path. The agent sees the prior attempt, checks the current policy, drafts the response, stops before sending, and leaves a receipt.
Same model.
Better system.
That is where Arc 3 starts.
Arc 1 was orientation. Arc 2 was inspection. Now we have to talk about engineering.
The system around the model is really the feedback loop around the model.
A model improves when the work creates feedback, and that feedback changes the next run.
That can happen in support, dashboards, onboarding, sales follow-up, pricing reviews, owner updates, content production, and almost every recurring workflow where the same mistake keeps showing up.
The real issue is often simpler: nothing around the model is learning.
That is the uncomfortable truth with AI.
When the system gives you the wrong answer, the easiest move is to blame the model. The harder move is to inspect the architecture around it.
Where was the policy supposed to live? Which source was supposed to win? What example should the agent have copied? What review signal should have changed the next run?
The model may have produced the sentence, but the system allowed the mistake. The user's job now is to architect the loop well enough that the next answer has a better chance of being right.
Think about a dashboard.
The old version is simple. A human checks revenue, compares a few dates, notices something weird, and sends an update.
The agentic version should pull current numbers, check source systems, compare today against yesterday, flag what changed, and stop when two sources disagree instead of smoothing over the conflict with a confident sentence.
Then the human reviews the answer.
If the human says, "Next time, always trust the property management system over the spreadsheet for booked revenue," that rule has to go somewhere.
If it does not, you have a dashboard with a chatbot on top.
If it does, you have the beginning of a feedback loop.
That is the difference.
The loop is the asset.
The people who win in the agentic AI era will have better feedback points built into the business.
They will define what went wrong, capture why it happened, update the system, and make the next run stronger.
Eventually, the best systems will act on that feedback automatically. A support issue updates the known-bugs note. A dashboard exception updates the source-of-truth rule. A failed onboarding step creates the next checklist item.
That is where this arc is going.
I am watching this with the current AI Strategy Academy work. The useful exercise is giving the agent one real workflow, letting it run up to a safe review gate, then saving where it got stuck.
The stuck point is the lesson. It shows you the missing policy, the missing example, the unclear owner, the tool with no clean door, or the decision that lived in someone's head.
That is why this arc has to get more technical.
Technical because the next level of AI work comes from engineering the loop.
Playbook
Here is the practical test.
Pick one real workflow where AI keeps repeating the same mistake.
A support reply. A weekly dashboard. A sales follow-up. A client onboarding step.
Then map the feedback loop.
1. What job is entering the system?
Start with the job, not the tool.
"Summarize this" is usually too vague.
"Read the latest customer message, check prior support history, draft a response, flag anything that needs approval, and stop before sending" is a job.
If the job is vague, the loop starts broken.
2. What prompt turns the job into clear work?
This is prompt engineering in the useful sense.
Not magic words.
Job description.
The prompt tells the model the role, outcome, sources, risks, return format, and stop sign.
People drop a half-formed thought into a machine and hope the machine interviews them silently. Sometimes it does. Often it guesses.
3. Can the system improve the prompt before the work starts?
That is meta-prompting.
Plain English version: the prompt before the prompt.
Instead of asking the AI to immediately do the work, you ask it to help shape the work first.
It interviews you: goal, audience, context, failure mode, quality bar, and what it should ask before acting.
That layer pulls clarity out of the human before the model starts producing.
4. What context does the model need at the exact moment of work?
This is context engineering.
Not "give it everything."
That is how you create an expensive junk drawer.
Context engineering is deciding what the model should see, when it should see it, and which source wins when two sources disagree.
For support, that might be customer history, current policy, refund rules, known bugs, and examples of good replies.
The model needs the right context.
5. What operating body is the model working inside?
This is harness engineering.
The harness is the environment around the model: instructions, tools, permissions, memory, rules, logs, and receipts.
A chat window is a thin version. A real agent system has more body: inbox, folders, calendars, databases, tests, notes.
But reach without rules is how you create anxiety.
The harness has to say what the agent can touch, what requires approval, what source to trust, and what proof to bring back.
6. How does the workflow run again with the correction included?
This is loop engineering.
The first run is not the win. The second run is where you learn whether the system improved.
The support agent should stop repeating the old policy. The dashboard should remember which source wins. The onboarding workflow should catch the missing contract before creating the next task.
If the answer is no, the loop did not close.
The human reviewed the work, but the system did not learn from the review.
That is the hidden failure.
The deeper version is autonomous loop engineering: the system notices repeated failure points, proposes updates, routes them for approval, and then changes the prompt, context, checklist, source rule, or tool path that caused the miss.
7. Which model should handle each part of the loop?
This is model routing.
The same workflow may need different intelligence at different points.
A cheaper model can classify tickets. A stronger model can reason through an unusual customer issue. A small model can extract dates. A separate reviewer can check the work.
The job now is to route the loop: draft here, review there, escalate when the risk changes.
8. Are your tools helping the loop, or breaking it?
This is systems engineering.
The more tools you add, the more places work can die.
Support inbox over here. CRM over there. SOP in a folder nobody opens. Dashboard in a tool the agent cannot query. Receipt in someone's memory.
Congratulations. You built a scavenger hunt.
The engineering job is to make the tools talk to each other enough that work can move. Sometimes that means consolidation. Sometimes it means an API, MCP server, script, export, webhook, or simple operating note.
The test is not whether the stack looks impressive.
The test is whether feedback from the work improves the next run.
Orientation
That is the Arc 3 frame.
Prompt engineering. Meta-prompting. Context engineering. Harness engineering. Loop engineering. Model routing. Systems engineering.
Build the loop around the model so the work gets clearer, safer, more useful, and more repeatable each time it runs.
That is what I am paying attention to now.
The next post is prompt engineering, but not in the generic "write better prompts" sense.
We are going to talk about giving an AI system a real job description: constraints, examples, edge cases, acceptance criteria, and a stop sign.
That is the first layer of the loop.
The AI Strategy Academy is where I am building this with operators live. The next cohort opens in July. If you want first notice when it opens, join the waitlist here:
For now, pick one AI workflow that keeps repeating the same mistake.
Then ask the real question:
Where is the feedback supposed to go?
Comment below and tell me the workflow.
I read every one.
- Brian