Claude 3.5 Sonnet vs Ornith-1.0-9B

Pick Ornith-1.0-9B if you want the stronger benchmark profile. Claude 3.5 Sonnet only becomes the better choice if you would rather avoid the extra latency and token burn of a reasoning model.

Ornith-1.0-9B is clearly ahead on the provisional aggregate, 52 to 40. The gap is large enough that you do not need to squint at the spreadsheet to see the difference.

Ornith-1.0-9B's sharpest advantage is in coding, where it averages 69.4 against 49. The single biggest benchmark swing on the page is SWE-bench Verified, 49% to 69.4%.

Claude 3.5 Sonnet is also the more expensive model on tokens at $3.00 input / $15.00 output per 1M tokens, versus $0.00 input / $0.00 output per 1M tokens for Ornith-1.0-9B. That is roughly Infinityx on output cost alone. Ornith-1.0-9B is the reasoning model in the pair, while Claude 3.5 Sonnet is not. That usually helps on harder chain-of-thought-heavy tests, but it can also mean more latency and more token spend in real use. Ornith-1.0-9B gives you the larger context window at 262K, compared with 200K for Claude 3.5 Sonnet.