Turn your agent's repeated mistake into a lint error
My coding agents kept repeating mistakes that were already written in their rules file. So now, when a mistake comes back, I turn it into a lint error and put the fix in the error message.
Once a week an agent command goes through the last 60 days of fixes, sorts them by mistake, and picks the strongest check it can for each one. The top 3 become small tickets, and each ticket ends with a check in CI.
Find the mistakes that came back
Start from evidence. Pull the last 60 days of:
- review comments that asked for a fix
- reverts and hotfixes
- lines you added to the rules file after an agent got something wrong
Group them into classes. A class is one mistake shape an agent can repeat: calling the wrong API, a missing guard, skipping a step. Keep a class only when it happened at least twice, independently: two different PRs or tickets. One bug reported twice counts once. If it happened once, I leave it out.
Then open the current code and make sure the mistake can still happen today. Sometimes a later refactor already made it impossible. Then it goes to the bottom of the list, and its rules-file line can go.
Rank what’s left:
- Classes whose rule is already written in the rules file and still came back. These are the ones that prove the doc isn’t working.
- Then by what it cost: reached production (a revert, a hotfix, a customer report) beats caught by a person in review, which beats caught by a check.
- Then by how often it happened.
Pick the highest level that works
For each class, go down this list and stop at the first level that can work:
- Architecture. Change the code so the mistake can’t be written. Give the job one owner module and delete the other ways of doing it.
- Types or lint. Bad code fails
tscor ESLint, and the error message names the fix. - A test. A spec fails when the behavior breaks.
- A doc line. Only for judgment calls, where nothing could fail.
Write one line for each level above the one you picked, saying why it doesn’t work. “Lint can’t see it, the bad value comes from the database” is a fine line. It also stops you from settling for a test because a test is what you’d usually write.
An example you can copy
Here’s a made-up one, rebuilt in a toy repo so the output below is real. A service has one module for outbound HTTP, src/lib/http.ts. It adds the auth header, a 5 s timeout and retries. The rules file says to use it. Agents keep writing this anyway:
// src/billing/refund.ts
import axios from 'axios';
export async function refund(chargeId: string) {
const res = await axios.post(`${process.env.PAYMENTS_URL}/refunds`, { chargeId });
return res.data;
}
and, in another PR, a plain fetch to the mail service. Both work in a quick test. Both skip the timeout and the retries.
You don’t need a custom rule for this. ESLint’s built-in no-restricted-imports, no-restricted-globals and no-restricted-properties all take a custom message. Put the fix in it, because that message is what the agent sees.
// eslint.config.mjs
import tsParser from '@typescript-eslint/parser';
const useHttp =
'Use `http` from src/lib/http.ts. It adds the auth header, a 5 s timeout and retries';
export default [
{
files: ['src/**/*.ts'],
languageOptions: { parser: tsParser },
rules: {
'no-restricted-imports': ['error', { paths: [{ name: 'axios', message: useHttp }] }],
'no-restricted-globals': ['error', { name: 'fetch', message: useHttp }],
'no-restricted-properties': ['error', { object: 'globalThis', property: 'fetch', message: useHttp }],
},
},
{
files: ['src/lib/http.ts'],
rules: { 'no-restricted-imports': 'off' },
},
];
The last block lets the one module that’s allowed to import axios keep doing it.
I ran it with ESLint 10.12 on that toy repo, at the commit with both bad files:
src/billing/receipt.ts
2:9 error Unexpected use of 'fetch'. Use `http` from src/lib/http.ts. It adds the auth header, a 5 s timeout and retries no-restricted-globals
src/billing/refund.ts
1:1 error 'axios' import is restricted from being used. Use `http` from src/lib/http.ts. It adds the auth header, a 5 s timeout and retries no-restricted-imports
✖ 2 problems (2 errors, 0 warnings)
The agent reads that in its own lint run and swaps the import.
The no-restricted-properties line is there because my first version missed globalThis.fetch("/x"). no-restricted-globals only catches the bare name. I found that by trying to get around my own rule, which is worth 2 minutes for every rule you add.
When the pattern is harder to describe (a call with the wrong arguments, a missing await on one specific function), write a custom rule. It’s one file with a create(context) function that visits the syntax tree and calls context.report. The same advice holds: put the fix in the message.
Prove the check on the mistake it’s for
Before the check merges, run it on the commit that had the mistake. If it passes there, it isn’t checking anything.
git worktree add --detach ../proof <bad-commit-sha>
cp eslint.config.mjs ../proof/
ln -s "$PWD/node_modules" ../proof/node_modules
(cd ../proof && npx eslint src) # must fail and name the mistake
npx eslint src # must pass on the branch
git worktree remove --force ../proof
A fresh worktree has no node_modules, so link the main one in. Without it, ESLint crashes on the parser import before it reads a single file.
Use a worktree so your branch stays as it is. Don’t use git stash for this: the stash is shared across worktrees, and an agent running in another one can pop your changes.
Paste both outputs into the PR body. That way the reviewer can see the check catch real bad code from your own history.
If there are existing occurrences, fix them in the same PR when there are 30 or fewer and the fix is mechanical. More than that and the PR gets too big to review; make the rule apply to new code only and file the cleanup separately.
Then delete the rule from the doc
Once the check is on your main branch, take the rule out of the rules file and add one row to a short “enforced rules” table instead: the rule in one sentence, the check that enforces it, and the commit it was proven on. What’s left in the rules file is stuff a linter can’t judge.