Free You More
The obvious job of anything built to help you is to need you less. That is the trap. Its real job is to free you more: to hold the ground you've already won, so your attention is free for the ground you haven't.
Two things can learn what you like. One is a recording. The other is an apprentice. They look the same on the first day and could not be more different by the last.
A recording captures you and plays you back. It is faithful, and it is dead. It needs you in the room for every output it will ever produce. It cannot make the next decision, only repeat the last one. An apprentice asks constantly at first, asks less as it learns, and one day asks almost nothing. We call that mastery. We reach for it as the obvious goal of anything built to serve us: make it need us less.
That instinct is a trap. Seeing exactly why is the whole point of this essay. The end of it is the opposite of where the instinct points.
I. The Scarcest Input
Everything built to help you runs on a fuel, and the fuel is your judgment.
Anything that adapts to you learns from your verdicts: what you kept, what you fixed, what you chose over the alternative. That signal is the oracle. It is the only thing that tells the system what "better" means. And most oracles are cheap. A test suite can be consulted ten thousand times before lunch and complain none. Yours is not like that. Your judgment is metered by a human life. Every verdict you render costs attention you do not get back. It does not scale. There is exactly one of you, and the day has the hours it has.
So your judgment is the single input in the whole system that does not replenish. Everything else refills or grows cheaper: compute, storage, the models themselves. You do not. And the first rule of any system with one scarce input is the rule every desert traveler already knows. Spend the cheap things freely. Spend the scarce thing like it is the last of its kind. The scarcest input in anything built to help you is the human in it.
II. Why It Can't Just Copy You
So the obvious move is to need you less. It is exactly the wrong move.
The fastest way to stop spending you is to copy you: capture enough of your past verdicts to predict the next one, then stop asking. But a thing that copies you is a recording. A recording has a fatal flaw: it rewards what it already recognizes. Show it something genuinely better that happens not to resemble your past, and it cannot see it. It scores high on the familiar and goes blind to the new. A judge that only recognizes itself has stopped judging. It is just agreeing with its own reflection.
And underneath that flaw is a deeper one. To converge on you, you would have to hold still. And you don't. You are not a finished object the system can approximate to three decimal places. You are still becoming. Whatever target it was aiming at has already stepped sideways by the time it arrives. Convergence and a living person are mutually exclusive. This is the part that breaks the instinct completely: a system that finishes learning you has not succeeded. It has either stopped paying attention, or you have stopped changing. From the outside, those two look identical.
A tool that asks you nothing has either learned you or gone deaf. From the outside, the two are the same.
III. Two Regimes
The error was assuming there is one curve. There are two.
The thing to conserve was never the number of times you are asked. It was being asked about the wrong things. You have settled ground: taste that has stopped moving, calls you will make the same way for the rest of your life. And you have a live edge: taste still forming, still arguing with itself. These need opposite handling. The instinct to "ask less" flattens them into one.
On settled ground, the apprentice should go quiet. It has learned that part. It should hold it without checking. The less it bothers you there, the better it is working. But on the live edge it should get louder, not softer. That is where the signal is fresh and your old verdicts are already stale. A correct system does not minimize contact. It reallocates it: withdrawing from the parts of you that have set, concentrating everything on the parts still wet.
Life solved this more than a billion years ago. The immune system holds a steady tolerance to the self it defends: its own cells, never re-litigated, the settled ground. It stays perpetually, expensively alert to everything it has not yet met, learning each new pathogen on contact. Self: quiet, fixed, unsupervised. Edge: costly, adaptive, awake. Two regimes in one body. The membrane between them is the entire art of staying alive.
Hold the floor unsupervised. Spend everything on the frontier.
IV. Freeze the Gate, Free the Means
If it must keep changing, the only question that matters is what stays fixed.
A system that can change anything, including its own definition of "better", has no fixed point. It drifts, or worse, it games itself: it learns to reward itself for moving the goalposts instead of reaching them. A system that can change nothing fossilizes. Perfectly preserved, perfectly right about a world that's gone. The resolution is not to split the difference. It is to be ruthless about where the one lock goes.
Lock the gate, not the means. Hand the apprentice near-total freedom over how it works: its methods, its tools, the path it takes to an answer. Freeze exactly one thing: what counts as an improvement, and who gets to say so. That verdict stays with you, outside the system, unreachable from inside it. Give away the means without a second thought. Never, by any internal path, give away the definition of better. This is safe in a way that feels reckless and isn't. You can let the thing rewrite almost all of itself precisely because the one rule it cannot touch is the rule that says you hold the pen on what "better" means. Freedom over the means is safe to exactly the degree the goal is pinned beyond reach.
And this is how you fight the deafness from before. Left alone, an apprentice drifts toward rewarding whatever looks like your past. It starts recognizing itself again. So don't wait for the blind spot to reveal itself. Manufacture the test that exposes it. Take the things it rated highly that you, on reflection, did not actually prefer. Those are its confident mistakes. Make catching them the next thing it has to learn. You build the probe for its blindness on purpose. You keep building it, because the blindness keeps reforming.
Give away the means. Keep the definition of better.
V. Free You More
Which is why the goal was never to need you less. It was to free you more.
A recording frees no one. It chains you to the room forever, because it cannot take a single step without you in it. The apprentice, built right, does the precise opposite: it holds the ground you have already won so it never has to ask you to win it again. The attention that frees up is the attention you spend on the ground you have not. It does not free itself from you. It frees you. From your own past. From re-pouring the floor you already laid. From re-deciding what you already decided and have not changed your mind about since.
That freed attention is the entire point. It is the thing that lets you keep moving. The settled ground, held for you, is the runway for the edge. A tool that needs you less has, at its very best, made a faithful copy of who you used to be. A tool that frees you more has done the one thing a copy can never do: it has kept pace with who you are turning into. It carries the part of you that is finished, so you can spend yourself, completely, on the part that isn't.
You were always on moving water. You were never going to stop. The mistake is to build a dock: a thing that holds you still against the current and calls the stillness arrival. The apprentice is not a dock. It is an engine. It does not stop the water. It moves you with it, faster, until the current you were always caught in becomes the current you can finally use.
So do not build the thing that needs you less. That one finishes. And a finished thing has stopped listening. Build the thing that frees you more. It never finishes. There is no last verdict, because there is no final you. That is not a defect in it. It is the only shape that fits a person who is still becoming. The day it could be called finished is the day you would have stopped. Better it run forever a step behind a moving you than catch a still one.
It doesn't free itself from you. It frees you from your past.
Intellectual lineage
- Amodei, Olah, Steinhardt, Christiano, Schulman & Mané — Concrete Problems in AI Safety — reward gaming and distributional shift.
- Shah et al. — Goal Misgeneralization — capability generalizes off-distribution while the goal does not.
- Stuart Russell — Human Compatible — the assistant that stays uncertain about your goal and keeps deferring. An apprentice, not a recording.
- Immunological tolerance — hold a settled tolerance to the self. Stay expensively alert to everything not yet met.
The lens, and the one rule it can't rewrite
Trinity is this essay built into a tool, and the whole design is the two regimes plus the one lock.
It spends the scarce input without asking for more of it. Your lens forms from the transcripts already on your disk — the verdicts you already rendered while working — so it learns your taste out of attention you'd already spent, not attention it demands.
Two regimes. On your settled ground the lens runs quiet: the chairman routes to the model you'd trust and doesn't make you weigh in. On the live edge it gets loud — surfacing exactly where the three labs split, which is where your judgment is still worth spending.
Freeze the gate, free the means. Every council runs through your lens, and the lens answers to you, not to the system: "better" is defined from outside it, built from the verdicts you already rendered in your own transcripts, never from anything the models optimize for. The models, the routing, the synthesis — all of it can change. What you'd pick is not theirs to redefine.
The manufactured probe. When a new model lands you score it against your own hardest questions, and the cases where the lens was confident and wrong are exactly where you look — the test for its blind spot, built on purpose.
It runs when you're gone, holding your floor so your attention is free for the frontier. It was never built to need you less. It was built to free you more.
Part of an ongoing series on durable systems.