Glossary

The competition

Unit — one task, one initial state, one demonstration, one seed. Both models are shown the same unit; each is scored on whether the goal holds at the end.

Skill — one of the 1 things a model is measured on: Sixteen tabletop tasks (two Franka arms, level 1). Each is scored as one success rate over its units.

Instance — the initial state a unit starts from: on RoboTwin the scene built from the unit's scene seed with the published embodiment. Published on every unit, never scored on its own.

Score — the mean of the skill success rates. Rendered as a percentage.

Duel — two models run against an identical list of units, drawn evenly across the skills.

Score margin — how far a challenger's average must exceed the reigning model's to take the crown: +3.0 pts.

Crown — what the winner of a duel holds. There is one.

Reigning model — the model currently holding the crown. It defends against every challenger in turn.

Baseline — a contest's own opening king, crowned unopposed at genesis and marked as such wherever it appears. This contest's: robotensor/vector-base.

Challenger — a submitted model waiting in the queue, or the non-incumbent side of a duel.

Queue — the ordered list of challengers. One entry per submission key; re-submitting sends you to the back.

Reign lineage — the public chain of predecessors for the crown, and the duel that ended each reign.

Void unit — a unit an infrastructure fault hit on either side. Excluded from both scores.

Paired outcome — won, lost or tied: whether the challenger succeeded where the reigning model failed, the reverse, or both did the same. Published as a diagnostic; never decides the crown.

The protocol

Demonstration — one continuous recording of the task, shown to the policy as its whole prompt. On this contest the policy is then scored from a different initial state.

Prompt — what the policy conditions on: the demonstration, materialized once per duel by the orchestrator and handed to both sides byte for byte. There is no language.

Initial state — the scene the policy is scored from: on this contest, the demonstration's own.

Materialized prompt — the demonstration bytes both sides were run from, produced once per duel before either side runs and published with the event by sha256, so anyone can check that both sides were shown the same thing.

Duel identifier — a hash of the specification version, the contest and both models' keys and revisions. Every unit's seed derives from it, so a draw is reproducible after the fact.

Protocol fingerprint — a hash of the specification file a duel was judged under. Published on every record.

The submission

Repository — a Hugging Face repository at a commit sha: the whole of a submission. It holds the manifest and the policy code and weights the manifest names, under 8 GiB.

Manifest — not this contest: submissions are weights, and nothing of them runs at the repository root (api not this contest: submissions are weights, and nothing of them runs): the policy class as module:Class, its constructor kwargs, its requirements file and the benchmarks it is meant for. An unknown key and a key given twice are refused.

Policy — the class the manifest names. It implements vector_policy.Policy: an action_type, set_demonstration(arrays, info), reset(seed) and act(observation). The orchestrator never imports it; a server inside the sandbox does.

Policy protocol — version 1: named numpy arrays and JSON fields over an authenticated socket, never a pickle. hello, prompt, reset, act, close.

Checkpoint — whatever weights the policy code loads, in whatever format. The orchestrator never reads them.

Sandbox — the container a policy runs in: network not this contest: submissions are weights, and nothing of them runs, a root filesystem not this contest: submissions are weights, and nothing of them runs, user not this contest: submissions are weights, and nothing of them runs, not this contest: submissions are weights, and nothing of them runs GPU, not this contest: submissions are weights, and nothing of them runs CPUs, not this contest: submissions are weights, and nothing of them runs of memory, not this contest: submissions are weights, and nothing of them runs processes, and nothing mounted from outside. Its only writable place, not this contest: submissions are weights, and nothing of them runs, is a tmpfs of not this contest: submissions are weights, and nothing of them runs that also runs code, so a policy may compile there at run time.

Base image — not this contest: submissions are weights, and nothing of them runs. Every submission's image is built from it at the pinned commit, and the event records the digest.

Revision — the commit sha the submission is pinned to, in full. Branch and tag names are refused, and so is an abbreviation.

Submission key — a short hash of repo@revision. The queue holds one entry per key.

Contest — a ladder (Vector: one demonstration from another scene, level 1). Each has its own king, queue, lineage, crown, skills and baseline. A submission is to one contest; contests are never compared.

Demonstration modality — what a contest's demonstration carries. This contest shows the frames, the action trajectory and the proprioception.

Withheld — the parts of a demonstration a contest keeps from its policies. On this contest: nothing.

Initial state — where a rollout starts. The contest scores a policy from a different initial state; whether the demonstration's actions can simply be replayed from there: no — the actions do not fit a different initial state.