New entry request: Hopper (G) 1.2 (a general-purpose Hopper, separate from Hopper)
#28
by ArkidMitra - opened
Hi, and thanks again for the re-run of Hopper and for maintaining the Index.
We'd like to request a new, separate row for Hopper (G) 1.2, a general-purpose version of Hopper. The
existing Hopper row (v1.1.1) should stay as it is.
- Weights:
HopitAI/hopper-g, revisiond60a1d6ca3f3fa25e5daddea16b8ba2b431ebcee(LoRA on Qwen/Qwen3.5-4B, same shape as Hopper). - Code: https://github.com/hopit-ai/hopper, tag
g-1.2.0(commit0204f92954f76fd8835f6050c271fc6e75ac3370). The serving code is
identical to Hopper 1.1.1: sameDecider.scoreentry point, same calibration map, long menus on by default. Only
the adapter differs, so the run is the same as your Hopper 1.1.1 run withHopitAI/hopper-gin place ofHopitAI/hopper. - What we measured on our own run of the 0.2 suite (all 40 benchmarks, A10G, same serving code): balanced raw
53.50 vs 52.74 for Hopper 1.1.1 (paired bootstrap +0.76, 95 % interval +0.55 to +0.98); balanced skill 38.07 vs
37.10. These are our numbers, not a claim about the board. - Licence: research and demo use only, as for Hopper (see the model card).
Happy to answer anything or change anything.
Hello, can you please score it yourself (following the instructioins here: https://github.com/apolinario/decision-index ) and submit as a PR to this same repo https://github.com/apolinario/decision-index ?