How AI Gets Smarter On Its Own Overnight — The Self-Improvement Design Behind Karpathy's autoresearch

What if, while you slept, an AI ran 700 experiments overnight — and you woke up to find it had discovered 20 improvements entirely on its own? It sounds like science fiction, but it actually happened. This is the story of autoresearch, an open-source project released in March 2026 by Andrej Karpathy, the AI researcher who led Tesla's self-driving team and worked at OpenAI.

The interesting part isn't the result itself. This project shows, in an unusually small and clear form, a single design principle for "making any project improve itself." And that principle applies not just to AI research but directly to ordinary business work as well. That's what we want to talk about today.

What autoresearch Actually Did

For months, Karpathy had been hand-tuning the code that trains an AI model, tweaking it little by little. Then he asked the obvious question: "Why am I doing this by hand? Why not hand the experiments off to an AI agent?" That question became autoresearch.

The structure is surprisingly simple. In practice, there are only three files.

Here's how it works. The agent edits train.py once, runs training for exactly five minutes, and grades the result with a single validation score (a value measuring how well the model predicts the next word — lower is better). If the score improves, it keeps the change; if it gets worse, it discards it and moves on to the next idea. It repeats this roughly 12 times an hour and about 100 times a night, with no human involved.

Karpathy summed up the key point in a single sentence.

"You're not directly editing the Python files the way a researcher would. You're programming the program.md markdown that gives the agent its context."

In other words, the human's job shifts from editing code to defining what to aim for, and how far it's allowed to change things. The result was clear. After a total of 700 experiments, the agent had discovered 20 optimizations on its own, and applying those improvements to a larger model cut training time by 11%. People began calling this approach the "Karpathy Loop".

There are signs this was more than a passing moment. In May 2026, Karpathy joined Anthropic's pretraining research team. You could read that as a sign that the "self-improvement loop" autoresearch demonstrated is closely aligned with what frontier AI labs actually care about.

The Real Innovation Isn't the Tool — It's the Self-Improvement Design

The autoresearch code is only about 630 lines. It didn't draw attention because it contained some remarkable technology. What's worth noticing is that it distilled the skeleton of a design that makes any system improve itself down to something this small. That skeleton comes down to four things.

1. Narrow the changeable surface to one place. autoresearch only allowed edits to a single file, train.py. If everything can be changed, nothing stays under control and nothing gets learned. Limiting the "place to freely experiment" to exactly one spot is the first move.
2. Define "better" as a single number. One metric — val_bpb — is the standard for every judgment. Because a number, rather than a person's gut feeling, decides whether things got better or worse, the loop can run without anyone present.
3. Run the loop on a fixed clock. One experiment takes five minutes; the keep-or-discard decision is immediate. You need a short, repeatable cycle to attempt 100 tries in a single night. A fast hundred beats a perfect one.
4. People set direction; machines do the running. What people contribute is choosing the goal, defining the metric, and designing the space of experiments — just those three things. Most of the actual execution can already be automated.

Notice that none of these four things require the word "AI model" at all. That's exactly why this design carries over directly to business work.

Bringing It Into Your Own Company's Work

A "system that improves itself overnight" isn't just for big corporate labs. Any work that repeats and whose success can be measured with a number can have these four steps laid on top of it.

• Customer-response messaging: Changeable surface = the wording of the first reply message. Metric = reply rate or booking-conversion rate. Loop = the AI generates message variations → tests them at small scale → adopts the better performer → repeats.
• Ad and landing-page copy: Changeable surface = a single headline. Metric = click-through rate. People only set the boundaries — such as "this tone is absolutely off-limits" — and leave the generating and comparing of variations to the machine.
• Internal work manuals (SOP): Changeable surface = one step of a specific procedure. Metric = processing time or number of errors. Test just one change each week, and if it improves things, fold it into the manual. But this is where most people get the first move wrong — if you set the metric as "processing time" alone, speed goes up but quality collapses. What you measure changes the entire outcome.

See the common thread? It's not about changing everything at once, but about picking one place to change and one number to measure, and running short cycles. Set up that one discipline, and the system starts improving on its own.

There is one trap you must call out, though. If you pick the wrong metric, you optimize the wrong thing. Set "shorter call time" as the sole metric, and agents start rushing customers off the phone. The moment a number becomes the goal, only the number improves while the thing that actually matters breaks down — this phenomenon is commonly known as Goodhart's law. That's why, in this methodology, a person's real job is to choose the right metric and draw the boundaries that must not be crossed. The machine runs the loop, but deciding which way it runs remains a human's job to the very end.

The first thing IROUMISM does when starting work with a client is exactly this. Which tasks to put on the loop, what to treat as the metric of "better," how far to let it change automatically and where a person must stop — drawing these boundaries together is where a self-improving system begins.

In Closing

The message autoresearch delivers is clear: self-improving systems come not from grand technology but from a small, clear design. One surface to change, one number to measure, a fixed loop, and a person to set the direction. With these four, a project starts getting better on its own.

The hard part is choosing "what to put on the loop." Which of your company's tasks best fits this self-improvement loop, and what metric to measure by so you avoid optimizing the wrong thing — we'll work through that with you.

Which of your company's tasks could go on a "self-improving loop" first? In a free 30-minute assessment, we'll help you find the first candidate.