01Report
Something breaks
A team reported that their agent had stopped replying. Before touching any code, I went to the logs.
Nothing matches. Try “case”, “resume” or “email”.
Case 01
What changed A single bad file can no longer take an agent down, and every provider error now names a precise reason.
Teams across the company built their AI agents on one shared platform, and I worked on it: the engine behind every turn, the studio where agents are made, and the gateways to chat and tools.
When something breaks
01Report
A team reported that their agent had stopped replying. Before touching any code, I went to the logs.
02Diagnose
A release had gone out that day, and it looked guilty. Instead of rolling it back on a hunch, I compared the broken agent with the healthy ones. The real cause was a single badly formatted file.
03Reproduce
I wrote a test that reproduces the bug and watched it fail. Then the fix: a bad file is refused when it is saved, and skipped safely when it is read, so it can’t take an agent down again.
04Ship
It was tested and reviewed before merging. Then I checked it on dev before production, so the fix didn’t break anything else.
05Watch
After the release, I watched the same logs that raised the alarm until they stayed quiet, then told the team it was fixed.
What the platform does
Share an agent without sharing your data. I built guest access across the services: per-guest permissions, separate credentials, isolated sessions.
Agents remember how you like answers, across chats.
People write in many languages. I built the resolver that picks each reply’s language, ran it in shadow mode, scored it on a labelled test set, then shipped it.
Owners teach agents with small instruction files. The studio checks each one on save and points to the line to fix.
Morning summaries without asking: a scheduled run posts a single clean message to the channel.
Agents reach real apps (logs, code, docs, travel) through a single tool gateway, covered in case 02.
Share this chat
1 ---2 name: query-logs3 description: Query logs: by user3 description: "Query logs: by user"4 ---
Every connection runs through the same gateway. That’s case 02 →
Working with the model
mainfixes and evals, each tested and reviewedfix/clear-provider-errorsClear failure reasonsEvery provider error used to show as the same vague failure. Each now maps to a precise reason, so people can see what went wrong.
eval/model-not-promptModel, not promptAgents ignored instructions and every prompt rewrite failed differently. A small A/B with the prompt held fixed pointed at the model, and the team changed the default.
fix/untrusted-fence-on-trimUntrusted text stays untrustedA path that shortened long tool results skipped the fence that marks them as data. It uses the fence now.
eval/stale-labelsWhen the answer key is wrongA score dropped after a rule change. The test labels still used the old rule, so I separated real regressions from stale labels and model noise.
fix/internal-context-off-channelsInternal instructions stay internalA filter stops the agent’s own instructions leaking into replies. Its first version matched words and wrecked good replies in testing, so I rebuilt it to match the leaked section’s shape.
feat/reply-language-resolverThe right reply languageThe resolver ran in shadow mode beside the old logic and was scored on a labelled test set before it went live.
How I ship a change like this
Things that broke
✕ a scheduled job stopped early and still logged success
✓ cause found: the step limit, not the rate limit
A scheduled job hit its step limit partway through and still recorded success. The chat app’s rate limit looked guilty; the real cause was the step limit. A job that says ok when it isn’t is worse than one that fails loudly.
✕ tool calls lost their credentials
✓ rolled back the same day; the missing test now exists
A gateway change refused calls without an identity, on the premise that every caller sends one. Some callers never did. The release was rolled back the same day, and the test that was missing covers exactly that caller: a call that arrives without an identity.
✕ an empty model name was rejected and re-sent every few minutes
✓ my mistake, found and fixed
A catalogue row was missing, so my change sent an empty model name. The engine rejected it, and the request was re-sent every few minutes. It was my mistake. I found it, fixed it, and now check for missing rows before I trust a lookup.
✕ the user’s message was never saved
✓ save the message the moment the turn is accepted
When a turn stopped before the model ran, the user’s message was never written, so it disappeared from the chat. The message is now saved the moment the turn is accepted, and tests cover both early-failure paths.
The day’s release looked guilty, and rolling it back would have felt productive. Comparing the broken agent with the healthy ones found the real cause instead.
Check a file when it is saved, and survive it when it is read. A bad file should cost a single skill, not a whole agent.
The language resolver ran beside the old logic and was scored on a labelled test set before it replaced anything.
A small A/B with the prompt held fixed showed the model was ignoring instructions. The team changed the default model on that evidence.