A multi-agent, zero-shot robotics planner that controls a robot straight from pixels — it sees, plans, and recovers from its own failures.

Gemma-4-31B × Cerebras · built for the Gemma 4 / Cerebras Hackathon
Yifan Kang
— add collaborators here —
Watch demo Code

60-second demo — closed-loop recovery, long-horizon sorting, and a Cerebras-vs-GPU speed race.

The idea

Real robots fail constantly. The only way to catch a failure is to look after every action — but that's impossible if perception is slow. TAMPire makes perception fast enough to verify each step, so the robot can notice a missed grasp from its own camera, replan, and recover — all zero-shot, with no task-specific training or teleoperation.

Three agents, one closed loop

Gemma perception

Gemma-4

Reads scene state directly from camera pixels and answers symbolic predicate queries.

in RGB frames · predicate query
out YES/NO verdict · object location · reason

TAMP planner

council

A planner drafts a plan; three critics check reachability, ordering & feasibility; a repair chair merges them.

in goal · predicate state
out ordered skill plan · replan on failure

Motor controller

OSC

Turns a symbolic skill into closed-loop motor commands on the real robot interface.

in skill + target · analytic grasp height
out end-effector / gripper / base commands
TAMPire agent architecture: who reads what, who emits what

See the agents work

Closed-loop recovery

It catches its own mistakes

A grasp misses — Gemma sees it from pixels, the planner replans, and the retry reaches the sink. Verified native success.

Long-horizon sort

Every step verified

Cereal, can, bread — each grasp confirmed by Gemma from the image before the next. Three of three, sorted.

Live multi-agent dashboards: perception (green), planner + goals (amber), motor controller (blue), and the message bus between them.

What it does

✓
Closed-loop recovery
catches a failed grasp from pixels → replans → native success
3/3
Long-horizon sort
fixed-base, real OSC; every grasp verified by Gemma before the next
~2.8×
Faster perception
same Gemma-4 model on Cerebras vs a GPU provider (throughput)

Why it works

Multimodal

State from pixels

Gemma-4-31B reads predicates — holding(obj), in(obj, bin) — directly from RoboCasa camera images, no perception pipeline to train.

Multi-agent

Plan, critique, repair

A council of a planner and three independent critics catches infeasible plans before the robot ever moves, then repairs them.

Speed

Verify every action

Cerebras serves Gemma at ~2.8× a GPU's throughput, so re-checking perception after each action — the key to recovery — stays cheap.

Physical AI

Real execution

Real operational-space control in simulation, reaching RoboCasa-native task success on recovery, sorting, and long-horizon stacking.

Built with

Gemma-4-31BCerebrasRoboCasa robosuiteMuJoCoPandaOmron / Panda OSC Python

A hackathon prototype. The recovery demo induces a first-attempt grasp failure to showcase the closed loop; scripted grasping is per-run stochastic; the speed comparison reports throughput on the same model. Code & details on GitHub.