Post-Astra Robotics

I'm writing this from Austin during CoRL 2026, my first CoRL. I'm an undergrad and only started doing robotics research recently, so I spent most of the week listening to people who have been in the field far longer than I have. A lot of what I heard was about agents.

On November 9, the first day of the main conference, Jim Fan's keynote had a slide about GPT-6 Astra, and within half an hour a clip of it was going around X. CoRL also ran its first-ever call for late-breaking results this year, with agentic robotics as one of the two topics.

In conversations between sessions, several researchers, some of them senior, seemed discouraged by how quickly LLMs are moving into robotics. If a coding agent can beat a VLA at manipulation, it's fair to ask what the last few years of policy learning were for, and what is left to work on. As someone deciding what to spend the next few years on, I've been asking the same thing.

The same day, Anthropic reported that Claude Fable 5.2 scores 64.9% on RoboDojo Sim, against 39.4% for Opus 5.5, 29.8% for GPT-6 Astra and 27.6% for Fable 5.1. These are Anthropic's own runs at maximum effort, and the announcement doesn't say whether 64.9% is a success rate or a partial-credit score. Either way, a general-purpose language model with no robot policy head now sits ahead of most of the systems our field built for robot control.

Anthropic hasn't released per-task results, so for most of this post I use the closest open data I could find. Agent as Policy (AGP) runs GPT-6 Astra inside a coding-agent harness and released all 2,100 of its RoboDojo episodes, logs and videos included. It scores 63.6, about where Fable 5.2 is.

I have two reactions to it, and I suspect neither is the popular one:

Most of the excitement I've seen comes with a sense that this came out of nowhere, but I think it follows fairly directly from what the benchmark measures and from what robotics already knew how to do. The second goes against the mood I described above, where people who work on robot learning read this result as the end of their approach. Manipulation has changed paradigms several times before (Fig. 2), and the older methods mostly stayed on as parts of the new systems. IK and motion planning outlasted the move to learning, and the agents now leading RoboDojo call them as tools. I expect learned policies to be kept in the same way. Working through why the first reaction holds also makes it clearer where the open problems are.

§1Look at what the benchmark asks for

RoboDojo has 42 simulated tasks in five categories: Generalization, Precision, Long-Horizon, Memory, and Open, with 50 episodes per task. The task list says more than the leaderboard does. A few examples:

• arrange number blocks so that they form a correct equation (Open)
• play tic-tac-toe (Long-Horizon)
• swap blocks around, keeping track of which is which (Memory)
• imitate a sorting sequence you were shown earlier (Memory)

In the equation task the robot has to read the digits, work out the arithmetic, decide which block goes where, then find those blocks and move them. Only that last part is really about motor control.

Today's VLAs have learned impressive action distributions; they fold laundry and open drawers they've never seen before. What they're missing is something like a brain. A VLA maps recent observations and an instruction to the next chunk of actions, and there's nowhere in that loop to hold a plan, track hidden state, or catch an arithmetic mistake. Tic-tac-toe mostly tests that missing piece, so it says little about how good the policy's hands are.

A frontier LLM approaches the benchmark from the other direction. It's already good at reasoning, memory and breaking a task into steps, and what it needs is a way to turn its decisions into motion.

§2Robotics already had the hands (for some tasks)

Anyone who has taken a classical manipulation course learns this early on, though it's easy to miss if you came into the field through imitation learning.

If you know where things are in 3D, a lot of manipulation is already solved. Given an object pose, you can sample grasps, solve inverse kinematics for the arm, plan a collision-free path and track it with a feedback controller. These methods are old and mostly hand-designed, and they work across objects and arms, which is what makes them general.

Some of this goes back a long way. Shakey was planning its actions with STRIPS in 1971. In the late 1980s, Lozano-Pérez's Handey system planned pick-and-place motions, grasps included, from a geometric model of the scene. RRT (LaValle, 1998) made motion planning practical for high-dimensional arms, and task-and-motion planning (Kaelbling and Lozano-Pérez, 2011, among many others) tied symbolic plans to geometric ones. Most of what the agent in Fig. 4 calls is a descendant of this work.

What they never had was something to drive them. Every new task needed someone to write the state machine by hand, deciding which object to pick first, where it goes, what counts as done and what to do when a grasp slips. That engineering cost is a big part of why the field moved toward end-to-end learning.

An agent takes over most of that cost. It looks at the scene, writes a program that calls perception, IK and the planner, runs it, looks again, and tries again if something went wrong. A VLA has to learn failure recovery from data, whereas an agent gets a basic version of it just by checking its own work.

Ken Goldberg made a similar argument in September, in an essay titled “Goosebumps”, where he calls agentic robotics “a third path between two dominant cultures in robotics.” I learned the learning culture first, and it was useful to be reminded how much of the other one is doing the work here.

It helps to look at what the agent actually does inside one of these episodes. AGP released the full event log for every episode. The model's private reasoning is encrypted, but the messages it writes and every tool call it makes are there. Below is layout 0 of the solve-equation task, which it finished in twelve tool calls.

The agent never outputs a joint command. It decides what to do in one sentence, and everything after that goes through tools that turn pixels into 3D points or plan a path to a pose. Before releasing the piece, the one step it can't undo, it measures again and works around a depth estimate it doesn't trust. This holds across the dataset: about 21,000 measure and 47,500 run calls over 2,100 episodes, roughly one in five reporting a failure the agent had to handle. In 260 episodes it also issued about 5,200 of those calls from JavaScript it wrote; the log keeps the calls but not the scripts.

One thing to keep in mind about this setup is that the AGP prompts also include a few hand-written hints per task, such as how wide to open the gripper before picking up a nut. Some human task knowledge is still in the loop, written as text instead of code.

§3Why now?

The architecture isn't new either. As frontier models got stronger, robotics trusted them with more of the stack (Fig. 6): first choosing among skills that were trained separately, then writing the program that calls perception and control, then improving a robot system on its own against a score, and most recently outputting the actions themselves. SayCan, Code as Policies and VoxPoser showed the idea working on short tabletop tasks in 2022 and 2023, with models much weaker than today's. With tools that are decades old and an architecture that is four years old, what changed has to be on the model side.

I'd point to four things that improved at roughly the same time.

• Coding. Driving a robot this way means writing and debugging real programs over many steps, which earlier models couldn't keep up for long.
• Spatial vision. The model has to locate the mug in coordinates accurately enough for IK to use, which is much harder than recognizing it.
• Reasoning budget. A few hundred robot actions take a lot of thinking even when compressed into code, and models can now spend that much without losing the thread.
• Self-improvement. When an attempt fails, the agent can watch the result, work out what went wrong and revise the code at test time. For a learned policy, learning from that failure means collecting recovery demonstrations and retraining; here it is an edit, and the fix carries over to the next attempt. Given a value function, the same loop runs without a person in it. ENPIRE reaches 99% on dexterous tasks this way, with agents rewriting the training code after every round (Fig. 6).

I don't know how much of this comes from generic pretraining. The labs don't say what they trained on, and the authors of An Unexpected Robot Policy point out that for a closed-weight model, contamination with benchmark-related material can't be ruled out. Either way, several separate abilities had to arrive at about the same time, which is part of why the jump looked so sudden.

§4Where it fails

Here is how AGP's 63.6 breaks down. Success means full task completion; the partial-credit score is what the leaderboard reports.

The harness matters about as much as the model. The same GPT-6 Astra scored 29.8% in Anthropic's runs and 29.0 (22.5% success) in An Unexpected Robot Policy, where it chose end-effector targets itself and a planner only executed them. Given measure and multi-step run, it more than doubles that. So the failures below are mostly about what the tools can't do.

Memory at 91% is what you'd expect. Precision at 44% is more surprising, since that's high for a system with no trained manipulation policy. The per-task numbers help explain it.

The 44% averages tasks near 90% with tasks near zero. Hitting a xylophone key needs precise positioning in free space; once the agent has located the key, it just moves there. Inserting a key needs precision in contact. Small pose errors make it jam, and fixing them takes fast force feedback the agent doesn't have. Peg-in-hole has been studied at least since Whitney's work on compliant assembly in 1982, where the usual answer was a wrist that gives a little under contact. Neither a coding agent nor a planner has that built in.

The task definitions make the difference concrete. Fasten screws, the one hands task the agent handles well, turns out to be closer to the xylophone. In the simulator the screws are hexagonal nuts that have to sit on posts to within 1 mm, and turning isn't checked, so it comes down to precise placement. Insert key needs the key upright to within 7°, pushed 2.5 cm into the lock, and then turned.

That makes me think the brain-versus-hands split I started with is a bit too coarse. A more useful distinction might be:

• Geometry-dominated manipulation: if the object can be localized and the motion planned, the agent and classical tools usually manage it.
• Contact-dominated manipulation: tight insertion, in-hand dexterity, deformables, and other tasks that depend on forces you can't see in an image. The agent doesn't have a good tool to call for these.

Long-Horizon, the weakest category at 28.8%, splits the same way (switch Fig. 8 to Long-Horizon). The agent does well when a task is long because it involves thinking, like sorting objects by class or playing a game. It fails when a task is long because one fine placement repeats many times, like eggs into an egg holder or pieces onto a stacking toy's pegs, and one slip anywhere fails the episode. In one fill-egg-holder episode it used 682 of its 700 allowed actions and still didn't finish. The plans look fine; the errors are small manipulation mistakes that compound.

All of this is simulation, and the real-robot evidence is thinner. The authors of An Unexpected Robot Policy stopped their real-robot trials of GPT-6 Astra after it repeatedly issued unsafe actions, some of which damaged hardware; in their reading, Astra understood the tasks more reliably than it executed them safely. Goldberg's group did better by keeping the model out of the control loop. Astra built a simulation of a sponge-wiping task from a short video and refined a controller against it, and the exported code ran on their real robot, though one trial knocked over the metal bar because its mass couldn't be inferred from video. In both cases the understanding transferred to hardware more easily than the contact did.

§5Is it fast enough?

Contact is one gap. Time is the other, and it's the objection I hear most often. The AGP logs report 300.6 hours of wall time for 14.3 hours of simulated robot motion, about 8.6 minutes per episode. Of the 300 hours, 208 were spent outside tool calls, in model inference and the agent runtime; simulation, rendering, and motion planning together took 93.

Per episode, that is about six minutes of agent time against 24.5 seconds of robot motion. The agent makes a median of 31 tool calls, and at each one it re-reads a growing context (1.45 million input tokens per episode, 96% served from cache) and writes a couple hundred tokens, about 5,900 output tokens per episode in all. I see three places to get that time down.

• Faster hardware. On Artificial Analysis, Cerebras already serves gpt-oss-120B at over 1,700 tokens per second, against a few hundred for the fastest frontier-tier models. I count 1.5–2× per year.
• Smaller models. By the densing law, the parameters needed for a fixed benchmark score halve about every 3.5 months. Decoding doesn't speed up in proportion, and routine calls can go to a small model while a large one handles surprises, so I count 1.5–3×.
• Fewer output tokens. In a footnote to Goldberg's essay, Jay Chooi reports Astra beating Fable 5.1 on a robot control task, 95% to 40%, with 6.2× fewer output tokens. I count 1.5–2.5× per year.

Multiplied together, that is 3.4× to 15× per year. Bringing six minutes down to 24.5 seconds takes about 14×, which lands somewhere between late 2027 and early 2029, for the agent's share alone; the 2.6 minutes per episode spent inside tools need their own speedup.

Some reasoning stays on the critical path even then, because recovering means looking at the result and deciding what to do. Robotics has an old answer to that. Brooks argued in 1986 for fast reactive layers that don't wait on a planner, and the three-layer architectures that followed (Gat, 1998) put a slow deliberative layer on top of a skill sequencer and a fast controller. The agent fits in that top layer, thinking in parallel while a skill runs and stepping in when it goes off track.

Does the LLM eventually become System 1 too, the fast layer that does dexterous control? I don't think so. A general model takes over a layer once it gets under that layer's latency threshold, and I expect that to happen in the middle of the stack, the pick-and-place decisions VLAs make today at a few hertz. A contact loop at hundreds of hertz needs about a thousand times more than the 14× above, and the same speedups make a small robot model faster too. Speed also isn't what's missing. The agent is equally slow on every RoboDojo task, yet it scores 96% on swap blocks and 0% on insert key, because knowing where the key goes doesn't tell you how to feel it into the slot. That comes from contact data, which web text and video contain very little of. So System 1 stays a separate, smaller model, back where §4 left off. What may change is who builds it: the agent can set up the simulator, collect the data and train the policy it will later call.

§6Is this the Bitter Lesson?

On the day of the Fable 5.2 release, Anthropic's Alex Albert posted that “robotics will fall to the bitter lesson too.” I think he's right, though probably not in the way the line is usually read in robotics. Sutton's argument is that general methods that scale with computation, mainly search and learning, eventually beat approaches built on hand-encoded human knowledge. It's a claim about methods, and it says nothing specific about LLMs or end-to-end training.

In robotics it usually gets read as a case for end-to-end learning: more demonstrations and bigger VLAs would eventually pick up reasoning as well. On this benchmark that hasn't happened. In the Open category, which holds solve equation and the language-conditioned tasks, no learned policy on the RoboDojo leaderboard scores above 25, while the top six agent entries score between 49 and 78. What worked is a model calling IK and motion planners, the hand-built components the end-to-end view expected us to discard.

Read the way Sutton wrote it, the result fits. Writing programs and searching over them at test time is a general method that scales with compute, and the planners sit outside it as tools. If robotics falls to the bitter lesson, it may fall with IK still in the loop. The more interesting question is where compute is best spent, whether on policy training, test-time reasoning, planning, or learning during deployment.

§7Why this is good news

High-level reasoning is becoming something you can get from an API. That removes a set of problems that robot learning was never in a good position to solve by collecting teleoperation data anyway, things like arithmetic, memory, task decomposition, and knowing what a charger is for.

What's left for us is mostly the physical side of manipulation, plus the problem of making all of this cheap and fast enough to use. Moravec pointed out in 1988 that the abilities we think of as intelligent, like arithmetic and games, are computationally easy, while perception and motor skill are hard. RoboDojo looks like the same observation almost forty years later: the agent solves the equation and then struggles to insert the key.

There's also a more practical reason to be optimistic, which is that agents are starting to do the slow parts of robotics research. The sponge-wiping simulation from Goldberg's essay took Astra under an hour to build from a short video, sponge deformation included. In Flip Anything, one of this year's CoRL late-breaking papers, a coding agent raises object-flipping success in simulation from 24.7% to 95.0% through its own iterations. Building simulators, tuning controllers and running ablations take up a large share of a robotics student's time, and all three are getting cheaper.

Learned policies don't become irrelevant either. Two entries on the RoboDojo agent track, including the second-ranked one, pair an agent with a learned model: GO-3 Nano Preview with a world-action model and PhysicalRSI with a VLA. The best insert-key score on the entire leaderboard, 48.4, belongs to PhysicalRSI. Goldberg's skill libraries can include VLA policies too. A reliable contact-rich skill is exactly the kind of tool an agent wants to call, so policy learning now has a new kind of user.

§8What's still open

Putting the earlier sections together, these are the problems I think stand between today's agents and general-purpose robots. A better language model helps with some of them, but none of them goes away on its own.

I came to Austin asking the same question as the people I talked to between sessions, whether there is still something left for me to work on. As the conference wraps up, I'm fairly sure there is. The key that jams and the eggs that slip were hard before Astra and are still hard, and the agents that fail at them can now help build the simulators and controllers it will take to fix them.