How the ladder works

One crown

There is one track and one crown. The crown has a reign, a public lineage of predecessors, and a queue of challengers. At genesis it is held by the organizer's own baseline checkpoint — BPP public checkpoint (austinpatel/libero, converted), published at robotensor/bpp-libero-genesis — crowned unopposed so that the first real entrant has something to beat. The baseline is marked as such wherever it appears.

Submit, queue, duel

You submit a model repository on Hugging Face — model.safetensors, config.yaml — at a pinned revision. It joins the queue. There is one entry per submission key: re-submitting replaces your entry and sends it to the back of the queue, which is what rate limiting looks like here instead of a daily quota.

When your turn comes, your model fights a duel with whoever holds the crown at that moment.

What a duel is

Both sides run the identical unit list. Not similar units, not a comparable sample — the same tasks, the same initial states, the same demonstration for each unit, the same seeds. The list is drawn evenly across the four axes: at the default heavy size that is 20 units per axis, 80 per side.

The challenger runs the whole list first, then the reigning model runs the same list. Each side's result on each axis is a success rate. The score is the mean of the four.

The crown rule

The crown transfers if and only if

challenger average ≥ reigning average + +3.0 pts

That is the whole rule. The margin is deliberate: an incumbent should not be displaced by noise, so a challenger has to show evidence of improvement, not evidence of parity.

Why copying the reigning model can never win

Submit an exact copy of the model that currently holds the crown. Both sides run identical units on identical seeds with identical weights, so they produce identical outcomes everywhere. The two averages are equal, the difference is zero points, and zero is short of the margin. The crown does not move — however good the copied model is.

No plagiarism detector is needed and none is used. The rule does the work. The same asymmetry is why a challenger cannot win by being merely similar: it has to be measurably better, on the same units, by the margin.

Void units

An infrastructure fault on either side — a simulator crash, a model that could not be loaded, a timeout the validator caused — marks the unit void. A void unit is excluded from both sides' scores, so neither model is penalised for the validator's problem. A duel with more than 10% of its units void is not published as a result at all; it is rerun.

Fresh units every time

Duel units are not drawn from a pinned suite. They are derived per duel: a deterministic identifier is computed from the specification version, the track, and both models' keys and revisions, and the tasks, initial states, demonstrations and seeds follow from that.

Three consequences. Every duel is a fresh draw, so there is nothing to overfit across duels. There is no secret test set to leak or to age out. And the draw is reproducible after the fact from published fields, so anyone can reconstruct exactly which units a duel ran.

What is published

Everything, as soon as a duel finishes:

• per-unit outcomes for both sides on every axis, with the perturbation each unit carried;
• three clips per unit — the demonstration, the reigning model's attempt, the challenger's;
• both sides' per-axis scores and the average, recomputed on the page from the unit list;
• the paired won/lost/tied counts, as a diagnostic;
• the protocol fingerprint the duel was judged under.

Next: the four axes.