Questions

Entering

Can I fine-tune at evaluation time? No. Parameters are frozen for the entire unit. There is nowhere in the loop for a gradient step, because the validator owns the forward pass.

Can I submit code? No. A submission is model.safetensors, config.yaml at a pinned revision, and the architecture is fixed to bpp_libero_v1. That is what lets both sides of a duel receive exactly the same prompt.

My training run produced a .ckpt. Now what? Convert it: icilval convert-ckpt writes the two files the gate expects, and icilval check-model runs the gate offline. What you submit has the commands.

How often can I resubmit? As often as you like, but there is one entry per submission key: a new revision replaces your entry and sends it to the back of the queue. That is the rate limit.

Can I submit a branch name as the revision? No. A revision is a commit hash. A mutable name would let the weights that hold a crown quietly stop being the weights that were scored.

The rules

How does the crown move? The challenger's average over the four axes must be at least the reigning model's average plus +3.0 pts, on an identical unit list. Nothing else decides it.

What are the won/lost/tied counts on a duel page? A diagnostic: on which units the two models differed. They do not decide the crown, and the page says so beside them.

Why can't a copy of the reigning model win? Identical weights on identical units with identical seeds produce identical outcomes, so the two averages are equal and the difference is zero points — short of the margin.

What happens if a unit crashes? If the fault was the validator's, the unit is void and excluded from both scores. If the model timed out or errored, the unit is a failure for that side and is recorded like any other outcome.

Is there language? No. none. The demonstration is the whole prompt.

How many demonstrations? 1. Always recorded from a different initial state than the one the model is scored from.

Evaluation

Which tasks are hidden? None of the task names — the benchmarks are open source. What is not public is which units a duel draws, the perturbed initial states, and the object pairings on the object axis, which were never published as training data.

Can I see the evaluation suite? There isn't one to see. Every duel derives its own units from a deterministic identifier, so there is no fixed test set to leak or age out — and the draw is reconstructible after the fact from published fields.

What is published about my submission, and when? The repository and revision, both sides' per-unit outcomes on every axis, three clips per unit, the per-axis scores and the average, and the diagnostic counts — all of it, as soon as the duel finishes, win or lose.

Who holds the crown at the start? The organizer's baseline checkpoint, BPP public checkpoint (austinpatel/libero, converted), crowned unopposed at genesis. It is marked Baseline wherever it appears, and the first entrant to beat it by the margin takes the crown.

The competition

Is there a private phase, a finalist round, or withheld results? No, no and no.

What would count as a real result here? A checkpoint that beats the baseline's average by the margin on a fresh draw across all four axes, with every per-unit outcome and every clip published for anyone to check.