# NYT Connections LLM Benchmark (extended version) (NYT Connections)

> Lech Mazur's NYT Connections benchmark gives a model 940 puzzles from the New York Times word game, each with up to four extra trick words, and asks it to sort the words into four groups of four in one try. It tests word knowledge, wordplay and resisting plausible decoys.

Canonical page: https://benchlm.ai/benchmarks/nyt-connections

- Category: [Reasoning](/reasoning)
- Last updated: November 13, 2026

## About NYT Connections

- Year: 2024
- Tasks: 940 NYT Connections puzzles, each with up to four added trick words
- Format: Sort 16 to 20 words into four groups of four, one attempt per puzzle, no feedback
- Difficulty: Wordplay, word knowledge and decoys that fit almost-but-not-quite
- Paper: [NYT Connections LLM Benchmark (Lech Mazur, GitHub)](https://github.com/lechmazur/nyt-connections)

Each puzzle is a Connections grid with up to four added words that fit none of its categories; the author checks that no added word fits any group in that puzzle. The model gets one attempt per puzzle, with no feedback and no "one away" hint, where a person on the Times site gets four mistakes. Since July 2026 the headline score uses quadratic-v1 scoring: a puzzle with g exact groups scores (g / 4) squared, so 0 to 4 groups earn 0%, 6.25%, 25%, 56.25% and 100%, and the score is the mean over all puzzles. Only runs that completed all 940 puzzles are on the main board. Many models are listed at several reasoning settings; we store one run per model on its profile (normally the highest setting) and name the others in its row note. The author also publishes a board for the newest 100 puzzles, as a check on puzzles that may sit in training data.

NYT Connections is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (94 models)

| Rank | Model | Creator | Score | Who ran it |
|------|-------|---------|-------|-----------|
| 1 | [Gemini 4 Carbon](/models/gemini-4-carbon) | Google | 99.3% | Run by NYT Connections |
| 2 | [Gemini 4 Argon](/models/gemini-4-argon) | Google | 98.6% | Run by NYT Connections |
| 3 | [GPT-6 Astra](/models/gpt-6-astra) | OpenAI | 98.4% | Run by NYT Connections |
| 4 | [GPT-6.1 Sol](/models/gpt-6-1-sol) | OpenAI | 97.8% | Run by NYT Connections |
| 5 | [Gemini 3.8 Flash](/models/gemini-3-8-flash) | Google | 97.4% | Run by NYT Connections |
| 6 | [Gemini 3.1 Pro](/models/gemini-3-1-pro) | Google | 97.4% | Run by NYT Connections |
| 7 | [Claude Fable 5.2](/models/claude-fable-5-2) | Anthropic | 97.3% | Run by NYT Connections |
| 8 | [GPT-5.5](/models/gpt-5-5) | OpenAI | 96.2% | Run by NYT Connections |
| 9 | [Gemini 3 Pro](/models/gemini-3-pro) | Google | 94.4% | Run by NYT Connections |
| 10 | [Claude Opus 5](/models/claude-opus-5) | Anthropic | 94.3% | Run by NYT Connections |
| 11 | [Gemini 3.7 Flash](/models/gemini-3-7-flash) | Google | 94.0% | Run by NYT Connections |
| 12 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | OpenAI | 93.8% | Run by NYT Connections |
| 13 | [Kimi K3](/models/kimi-k3) | Moonshot AI | 93.6% | Run by NYT Connections |
| 14 | [Kimi K3.1 Pro](/models/kimi-3-1-pro) | Moonshot AI | 93.4% | Run by NYT Connections |
| 15 | [Muse Wonder](/models/muse-wonder) | Meta | 93.4% | Run by NYT Connections |
| 16 | [DeepSeek V4.1 Flash 1017](/models/deepseek-v4-1-flash) | DeepSeek | 92.8% | Run by NYT Connections |
| 17 | [Claude Fable 5](/models/claude-fable) | Anthropic | 92.7% | Run by NYT Connections |
| 18 | [Gemini 3.5 Flash](/models/gemini-3-5-flash) | Google | 92.6% | Run by NYT Connections |
| 19 | [Claude Opus 4.6 (Adaptive)](/models/claude-opus-4-6-thinking) | Anthropic | 92.1% | Run by NYT Connections |
| 20 | [GPT-5.4](/models/gpt-5-4) | OpenAI | 91.3% | Run by NYT Connections |
| 21 | [Claude Opus 4.8](/models/claude-opus-4-8) | Anthropic | 91.1% | Run by NYT Connections |
| 22 | [GPT-6 Sol](/models/gpt-6-sol) | OpenAI | 90.1% | Run by NYT Connections |
| 23 | [Claude Fable 5.1](/models/claude-fable-5-1) | Anthropic | 90.0% | Run by NYT Connections |
| 24 | [Grok 4.20 Multi-agent](/models/grok-4-20-multi-agent-beta) | xAI | 89.6% | Run by NYT Connections |
| 25 | [Gemini 3.6 Flash](/models/gemini-3-6-flash) | Google | 89.0% | Run by NYT Connections |
| 26 | [Claude Opus 5.5](/models/claude-opus-5-5) | Anthropic | 88.5% | Run by NYT Connections |
| 27 | [Qwen3.8 Max](/models/qwen3-8-max) | Alibaba | 88.3% | Run by NYT Connections |
| 28 | [Grok 4.1 Fast (Reasoning)](/models/grok-4-1-fast-reasoning) | xAI | 87.4% | Run by NYT Connections |
| 29 | [Kimi K2.6](/models/kimi-2-6) | Moonshot AI | 87.2% | Run by NYT Connections |
| 30 | [GPT-6.1 Terra](/models/gpt-6-1-terra) | OpenAI | 86.8% | Run by NYT Connections |
| 31 | [Grok 4.20](/models/grok-4-20-beta) | xAI | 85.4% | Run by NYT Connections |
| 32 | [Qwen3.7 Max](/models/qwen3-7-max) | Alibaba | 85.1% | Run by NYT Connections |
| 33 | [Muse Spark 1.3](/models/muse-spark-1-3) | Meta | 85.1% | Run by NYT Connections |
| 34 | [Muse Spark 1.1](/models/muse-spark-1-1) | Meta | 84.9% | Run by NYT Connections |
| 35 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 83.6% | Run by NYT Connections |
| 36 | [Hy4](/models/hy4) | Tencent | 83.5% | Run by NYT Connections |
| 37 | [Gemini 3 Flash](/models/gemini-3-flash) | Google | 83.1% | Run by NYT Connections |
| 38 | [Claude Sonnet 5.5](/models/claude-sonnet-5-5) | Anthropic | 80.5% | Run by NYT Connections |
| 39 | [Grok 4.6](/models/grok-4-6) | xAI | 80.0% | Run by NYT Connections |
| 40 | [Grok 4.5](/models/grok-4-5) | xAI | 79.9% | Run by NYT Connections |
| 41 | [Muse Spark 1.2](/models/muse-spark-1-2) | Meta | 79.2% | Run by NYT Connections |
| 42 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | OpenAI | 78.4% | Run by NYT Connections |
| 43 | [Muse Spark Open](/models/muse-spark-open) | Meta | 78.4% | Run by NYT Connections |
| 44 | [GLM-5.1](/models/glm-5-1) | Z.AI | 77.7% | Run by NYT Connections |
| 45 | [Grok 4.7](/models/grok-4-7) | xAI | 76.8% | Run by NYT Connections |
| 46 | [Step 5 Preview](/models/step-5-preview) | StepFun | 76.6% | Run by NYT Connections |
| 47 | [Claude Sonnet 5](/models/claude-sonnet-5) | Anthropic | 75.1% | Run by NYT Connections |
| 48 | [Qwen3.7 Plus](/models/qwen3-7-plus) | Alibaba | 74.8% | Run by NYT Connections |
| 49 | [GLM-5 (Reasoning)](/models/glm-5-reasoning) | Z.AI | 74.8% | Run by NYT Connections |
| 50 | [GLM-5.2](/models/glm-5-2) | Z.AI | 74.3% | Run by NYT Connections |
| 51 | [GLM-5.3](/models/glm-5-3) | Z.AI | 74.2% | Run by NYT Connections |
| 52 | [Qwen 3.6 Max (preview)](/models/qwen3-6-max-preview) | Alibaba | 74.1% | Run by NYT Connections |
| 53 | [Gemma 4 31B](/models/gemma-4-31b) | Google | 70.6% | Run by NYT Connections |
| 54 | [Kimi K2.5 (Reasoning)](/models/kimi-k2-5-reasoning) | Moonshot AI | 69.9% | Run by NYT Connections |
| 55 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | OpenAI | 69.4% | Run by NYT Connections |
| 56 | [GPT-6 Luna](/models/gpt-6-luna) | OpenAI | 68.7% | Run by NYT Connections |
| 57 | [Hy4 preview](/models/hy4-preview) | Tencent | 68.2% | Run by NYT Connections |
| 58 | [Claude Haiku 5.5](/models/claude-haiku-5-5) | Anthropic | 65.7% | Run by NYT Connections |
| 59 | [MiniMax M3](/models/minimax-m3) | MiniMax | 65.1% | Run by NYT Connections |
| 60 | [GPT-5.4 mini](/models/gpt-5-4-mini) | OpenAI | 61.8% | Run by NYT Connections |
| 61 | [Gemini 3.5 Flash-Lite](/models/gemini-3-5-flash-lite) | Google | 60.4% | Run by NYT Connections |
| 62 | [Qwen3.6 Plus](/models/qwen3-6-plus) | Alibaba | 60.3% | Run by NYT Connections |
| 63 | [Qwen3.5 397B (Reasoning)](/models/qwen3-5-397b-reasoning) | Alibaba | 58.9% | Run by NYT Connections |
| 64 | [Grok 4.3](/models/grok-4-3) | xAI | 55.2% | Run by NYT Connections |
| 65 | [Qwen3.8-27B](/models/qwen3-8-27b) | Alibaba | 54.5% | Run by NYT Connections |
| 66 | [Claude Opus 4.5 Thinking](/models/claude-opus-4-5-thinking) | Anthropic | 52.5% | Run by NYT Connections |
| 67 | [Qwen3.5-122B-A10B](/models/qwen3-5-122b-a10b) | Alibaba | 51.7% | Run by NYT Connections |
| 68 | [Claude Opus 4.5](/models/claude-opus-4-5) | Anthropic | 49.4% | Run by NYT Connections |
| 69 | [Qwen3.5-27B](/models/qwen3-5-27b) | Alibaba | 47.9% | Run by NYT Connections |
| 70 | [Claude Sonnet 4.6](/models/claude-sonnet-4-6) | Anthropic | 44.8% | Run by NYT Connections |
| 71 | [Qwen3.7 Flash](/models/qwen3-7-flash) | Alibaba | 43.8% | Run by NYT Connections |
| 72 | [Claude Opus 4.6](/models/claude-opus-4-6) | Anthropic | 43.2% | Run by NYT Connections |
| 73 | [Qwen3.6-35B-A3B](/models/qwen3-6-35b-a3b) | Alibaba | 41.6% | Run by NYT Connections |
| 74 | [Hy3](/models/hy3) | Tencent | 41.2% | Run by NYT Connections |
| 75 | [Step 3.7 Flash](/models/step-3-7-flash) | StepFun | 39.7% | Run by NYT Connections |
| 76 | [Claude Opus 4.7 (Adaptive)](/models/claude-opus-4-7-adaptive) | Anthropic | 39.0% | Run by NYT Connections |
| 77 | [Claude Sonnet 4.5 Thinking](/models/claude-sonnet-4-5-thinking) | Anthropic | 37.3% | Run by NYT Connections |
| 78 | [DeepSeek V3.2 (Thinking)](/models/deepseek-v3-2-thinking) | DeepSeek | 36.7% | Run by NYT Connections |
| 79 | [Claude Sonnet 4.5](/models/claude-sonnet-4-5) | Anthropic | 35.8% | Run by NYT Connections |
| 80 | [MiMo-V2.5-Pro](/models/mimo-v2-5-pro) | Xiaomi | 34.4% | Run by NYT Connections |
| 81 | [MiniMax M3.1 Flash](/models/minimax-m3-1-flash) | MiniMax | 28.3% | Run by NYT Connections |
| 82 | [Mistral Large 4](/models/mistral-large-4) | Mistral | 27.4% | Run by NYT Connections |
| 83 | [MiMo-V2-Pro](/models/mimo-v2-pro) | Xiaomi | 25.8% | Run by NYT Connections |
| 84 | [MiniMax M2.7](/models/minimax-m2-7) | MiniMax | 24.7% | Run by NYT Connections |
| 85 | [Hy3 Preview](/models/hy3-preview) | Tencent | 17.2% | Run by NYT Connections |
| 86 | [Trinity-Large-Thinking](/models/trinity-large-thinking) | Arcee AI | 16.5% | Run by NYT Connections |
| 87 | [Claude Haiku 4.5](/models/claude-haiku-4-5) | Anthropic | 14.3% | Run by NYT Connections |
| 88 | [Mistral Medium 3.5 128B](/models/mistral-medium-3-5-128b) | Mistral | 12.9% | Run by NYT Connections |
| 89 | [Grok 4.1 Fast](/models/grok-4-1-fast) | xAI | 12.3% | Run by NYT Connections |
| 90 | [Claude Opus 4.7](/models/claude-opus-4-7) | Anthropic | 10.8% | Run by NYT Connections |
| 91 | [DeepSeek V3.2](/models/deepseek-v3-2) | DeepSeek | 8.2% | Run by NYT Connections |
| 92 | [Gemini 3.1 Flash-Lite](/models/gemini-3-1-flash-lite) | Google | 8.2% | Run by NYT Connections |
| 93 | [Llama 4 Maverick](/models/llama-4-maverick) | Meta | 8.0% | Run by NYT Connections |
| 94 | [Mistral Large 3](/models/mistral-large-3) | Mistral | 7.5% | Run by NYT Connections |

## FAQ

### What does the NYT Connections benchmark measure?

Whether a model can solve the New York Times Connections word puzzle: sort the words into four groups of four that share a hidden link. This version adds up to four trick words per puzzle that fit no group, and allows one attempt with no feedback, so it rewards wordplay, broad word knowledge and resisting near-misses.

### Which model leads NYT Connections?

Gemini 4 Argon at high reasoning leads the extended version with 98.6%, ahead of GPT-6 Astra at max reasoning (98.4%) and GPT-6.1 Sol at max reasoning (97.8%). The best open-weight model is Kimi K3 at 93.6%.

### Why do some models appear several times?

The author runs many models at several reasoning settings, and the setting changes the score a lot: GPT-5.2 scores 83.6% at xhigh reasoning and 14.5% with reasoning off. Each model profile on BenchLM shows one run, usually the highest setting, and its note lists the other runs.

## Compare Top Models on NYT Connections

- [Gemini 4 Carbon vs Gemini 4 Argon](/compare/gemini-4-argon-vs-gemini-4-carbon)
- [Gemini 4 Argon vs GPT-6 Astra](/compare/gemini-4-argon-vs-gpt-6-astra)
- [GPT-6 Astra vs GPT-6.1 Sol](/compare/gpt-6-1-sol-vs-gpt-6-astra)
- [GPT-6.1 Sol vs Gemini 3.8 Flash](/compare/gemini-3-8-flash-vs-gpt-6-1-sol)