Skip to content

Under maintenance. Some results may be incomplete or inaccurate.

Robotensor
All pages21

Horizon · Reference

Questions

The questions a reader of this site arrives with, answered in a sentence or two.

Look-up page — built for scanning

Reading this site

Why does this epoch have no scores? It has only opened. epoch open publishes the commitment, the cutoff and the entries the register held at it; the scores come with the record that closes the epoch. Until then there is nothing scored to show, and nothing is estimated in its place.

Why does the page say a value is "not published"? Because the record it is reading does not carry it. A store holds records from every version of the competition that wrote it, and several fields — pins, pool_manifest, scoring_config, the overall score, the voids — arrived over time, or went away again. The gap is shown rather than filled.

What is a dry run, and why is it listed apart? It is a rehearsal of a close, signed as its own record and marked as one. It is no week the competition ran, so it is never an epoch's result and never the latest epoch. Its numbers are shown on its own page because it was published.

Why is this epoch shown as not verified? Either a signature did not check out, or the store's index and a record disagree, or — the case signing at open exists to catch — the close record reveals a secret for a different commitment, or under a different config, than the open record signed. The page says which. An unverified record is shown as unverified rather than hidden.

Why is a clip or a result marked unsigned? Because the record that covers its epoch is from before record version 2, which is the version that began signing a digest for each bundle and each result. The file may be exactly the one that was run; nothing here can tell you so, and the mark says as much.

Why do two epochs' numbers not compare? Look at the record version and the config. The scoring rule changed at version 2 — paired units, a paired standard error, and no winner at no wins — and the scoring config can change between epochs in any case. Both are in each epoch's record.

Entering

Can I submit code? No. A submission is weights: a Hugging Face repository at a commit with the transformer, your normalisation statistics and a zerowam.yaml of inference settings. Every model in an epoch is served by the same runtime. Entering is the whole of it.

How do I know my repository will be accepted? Run horizon-runtime-zerowam check --weights on it. It reads file headers only, takes seconds and needs no GPU, and the organiser runs the same command on the copy they fetch — so what it says on your box is what decides your entry.

Why does the intake want a Hugging Face token? To ask the Hub whether you can write to the repository you named. That is what stops somebody else entering your repository under their name. The token fetches the repository and is then gone: not stored, not logged, not published.

I submitted the same commit twice. Is that two entries? No — one entry, with the time it first arrived. Identical weights are one entry whoever sends them, and the earlier one keeps the place.

Does submitting a small variation of the leader get ahead of it? Only by actually scoring higher. A submission's score is the mean of its axis scores and nothing else, so a copy scores what the model it copied scores; and where the two tie exactly, the tie goes to the earliest submitted_at, which is the leader's.

Checking the numbers

Can I check this myself? Yes, and with nothing installed but the competition's own package — no simulator, no fork, no GPU. store verify checks every signature in a store; epoch derive rebuilds an epoch's units from its revealed secret and the record's own recipe. Verification has the first command; the second takes the epoch's published record and the secret it reveals.

Where do I get the organiser's public key? From somewhere other than the store you are checking. A store rewritten and re-signed passes a check against the key it names itself; only a key from outside says whose records these are. Where this deployment pins one, the printed command carries it.

Why is there no --public-key in the command I am shown? Because this deployment pins none, or what it pins is not a hex ed25519 key. The command is printed as it runs — no blank to fill — and the note beside it says what the check is then worth.

Why can I not re-derive an older epoch? Its record carries no units recipe. Before the competition recorded one, the task list came from the forks at open and was never written down, so epoch derive refuses the record rather than deriving other units. The page prints no command for one, and says why.

The demonstrations are in the epoch directory, not the record. Why should I trust them? Only as far as the checks go, and the page says how far that is: the directory has to agree with the record on the secret commitment, the config's sha256 and the units, and from record version 2 every file shown is hashed here and held to the digest the record signs for it.

Questions · Docs · Robotensor competitions