Vector · The guide
The skills
What each scored skill asks, how a unit of each is built, and what every skill shares.
2 min read
Everything in the competition is built out of one object: the unit. A unit is one task, one initial state, one demonstration and one seed. Both policies see the demonstration, both start from the initial state, and both are scored on whether the task was done at the end. The skills differ in what the task is; the benchmark, the suite and the scene are the same.
The skill Sixteen tabletop tasks (two Franka arms, level 1)
RoboTwin's 16 pick-and-place and tool tasks of the level benchmark, dealt evenly over a duel's units. RoboTwin's expert produces the demonstration at evaluation time in one scene and its own check decides the outcome in another. Its tasks are the vector_level1 suite of the robotwin benchmark, capped at 800 steps, and a unit is shown through head_camera, left_camera, right_camera. The skill's score is one success rate over all of its units, and nothing below it — no task, no scene — is scored on its own. The sixteen tasks are dealt evenly over a duel's units, so no task decides a crown by itself.
What every skill shares
- One demonstration, always of the same task; the policy is then scored from a different initial state. The demonstration is the whole prompt.
- No language. The task is never named to the model. The task names are public — the benchmarks are open source — but nothing hands them to the policy.
- Nothing reachable but the demonstration. A policy's only inputs for a unit are the demonstration and its own observations: no network, no store, no other side. What it does with them inside its budgets — attend over them, fit to them, replay them — is not inspected.
- The orchestrator owns the loop — it materializes the demonstration, drives the environment and scores the outcome, for every skill. An entrant's policy code answers inside its sandbox, with the same policy on every skill.
- Both sides see the same unit. Same initial state, same demonstration, same seed.
How the score is built
Each skill is a success rate over its units, excluding any that were void. The score is the mean of the skill rates. A duel page shows every skill's rate for both sides, with the difference in points beside each, and then the average row — the one comparison the crown rule makes — in the context of the rates that produced it.
The draw, at the default duel size
- Default duel size
- launch
- Units per skill, default duel
- 160
- Vector: one demonstration from another scene, level 1: units per side, default duel
- 160
- Selectable duel sizes
- smoke, light, launch, full
Rendered from spec.json at build time — the same file the orchestrator reads.
Skills are meant to accumulate: adding one adds a row to every ledger and bumps the specification version so no earlier draw is confused with a later one.
Next: what you submit.
