ENCORE
Paper ↗Code ↗

Few-Shot Agentic Discovery of Manipulation Strategies

Yifan Kang1,2 · Zihan Wang2 · Zhiwen Fan3 · Bangya Liu2,4

1MIT   22077AI   3Texas A&M University   4University of Wisconsin–Madison

WRL @ NeurIPS 2026 · Oral CoRL 2026 Workshop on Agentic Robotics
Paper ↗Code ↗BibTeX
MIT2077AITexas A&M UniversityUniversity of Wisconsin–Madison
Overview, 99 s, with narrationSim + hardware

Highlights

Sealed evaluations, no success signal at test time

96.3%LIBERO-PRO, 60 perturbed tasksvs 89.3% for ASPIRE on the same model
29/60tasks where the first program already works1 / 60 without demonstrations
0/750episodes solved without demos when the goal is unstatedRoboDojo, 3 independent runs
9/10real robot, cube handover and cup inversion5 demonstrations each

Summary

A robot instruction says what to do, not how. ENCORE gives a coding agent a few demonstrations as evidence to read, not as training data. The agent writes a policy program, refines it over a few rollouts, and freezes it before a sealed test that never shows it the success signal.

Method

ENCORE overview: the demonstration pack feeds an agentic coding loop whose programs run in simulation and on a bimanual robot
The agent reads the pack, writes a program, runs it, observes, refines, and freezes it.Figure 1

01Distill the demonstrations

A deterministic builder turns each demonstration into a pack: multi-view keyframes at the demonstrator's gripper events, frame strips around each contact, and the full trajectory. It selects frames; it never labels them.

02Write, run, refine

A fresh coding agent studies the pack with analysis code it writes itself, then writes a policy program against a narrow API: sense, perceive, act. It refines the program over 15 development rollouts and checks its own grasps and placements.

03Freeze, then evaluate once

The chosen version is frozen and its hash recorded. A coordinator evaluates it once on 50 sealed initial states, judged only by the benchmark's own success check.

Results

01Ahead of prior agents

On LIBERO-PRO's position and instruction perturbations, the frozen programs succeed on 96.3% of sealed episodes, against 89.3% for ASPIRE run on the same language model and 71.7% as published.

ObjectGoalSpatialMean
PosTaskPosTaskPosTask6 columns
π0.5
17
1
38
0
20
1
12.8
CaP-Agent0
22
18
26
17
12
14
18.2
ASPIRE (opus-4.6)
98
95
81
45
51
60
71.7
ASPIRE (opus-5)
100
100
86
86
84
80
89.3
ENCORE, no demos
99.6
100
92
86.6
95.8
95.2
94.9
ENCORE, 3 demos
100
100
93
90
97.8
97
96.3
Sealed success (%) per suite and axis, 10 tasks × 50 episodes each. π0.5, CaP-Agent0 and ASPIRE (opus-4.6) as published; ASPIRE (opus-5) is our rerun of its released code.LIBERO-PRO

02Demos set the first program

With demonstrations, the first program already works in 29 of 60 tasks; without them, in one. The demonstrations change how development starts, and the agent reaches its first success in fewer versions.

Success by program version

025507510015101520 41% 1% 3 demos no demos program version

Where the demonstrations save

version of first success
7
45
versions before freezing
10
44
simulator steps
18
42
agent hours
20
40
development episodes
26
32
cost
27
33
tasks where 3 demos need lessneed more
Left: mean development success of each task's current program at version k. Right: in how many of 60 tasks the agent with demonstrations needs less or more.60 tasks

03A strategy the search missed

The sentence asks for the bottom drawer; the demonstrations open the middle one. Without them, the agent concludes the handle cannot be engaged. With them, it copies the demonstrated downward drag and opens the drawer in every run.

no demonstrations0 / 3 runs0, 0, 0 of 50
3 demonstrations3 / 3 runs50, 47, 50 of 50

04Composing demonstrated skills

On rewritten instructions, each task gets two packs and neither demonstrates its goal. The sentence decides which half of each pack to reuse.

Two demonstrated skills recombined into a new task, in two cells that share a pack
Both tasks score 50 / 50 with the two packs.Recombination

05When the goal is unstated

"Build a tower using the wooden blocks and boards" does not say which tower. On five such RoboDojo tasks, no program written without demonstrations succeeded in any of three independent runs: 0 of 750 episodes. The demonstration-free agent solves the manipulation but cannot tell what to build.

Successes out of 50, first run per task; the images-only arm was run on the contact-sensitive tasks only. Make kong is scored with the grounding model of the later runs. Repeated runs are in the paper's appendix.RoboDojo, bimanual

Real robot

Same loop, same API

On a bimanual Trossen WidowX AI with four RealSense cameras, five demonstrations per task: cup inversion 9 / 10 and cube handover 9 / 10, in sealed trials judged by a person who did not develop the program.

Cup inversion: the demonstration as the pack presents it, the agent's reading, and the frozen program
From five demonstrations to a frozen program.Cup inversion
Goal states: the cube passed and placed in the bowl; the cup upside down
Goal states of the two tasks.Hardware

BibTeX

@article{kang2026encore,
  title   = {ENCORE: Few-Shot Agentic Discovery of Manipulation Strategies},
  author  = {Kang, Yifan and Wang, Zihan and Fan, Zhiwen and Liu, Bangya},
  journal = {arXiv preprint arXiv:2609.37359},
  year    = {2026}
}