Crosshair LLM Benchmark

Wouldn't it be nice to have a board game to evaluate models against that isn't in the training data? Well, I introduce to you Crosshair, which I invented on October 25, 2026. Should be safe for a few generations yet!

Crosshair is a 7×7 connection-and-flip board game built specifically to work well for a benchmark like this:

• Zero training contamination: Crosshair didn't exist when these models were trained, so no games, opening books, or strategy guides appear in pretraining data. Every move has to be worked out from the four-sentence ruleset in the prompt.
• No draws and fast games: Red connects top to bottom; Blue connects left to right. Orthogonal sandwiches and 2×2 diagonal crosscuts flip enemy stones in place and can chain across the board, so draws cannot happen and every game ends by ply 49 (usually in 25 to 40 plies).
• Small branching factor (~8 to 12 legal moves per turn): Playing at (c, r) restricts your opponent's next move to empty squares in Row r or Column c (unless that cross is full, which gives them a free move anywhere). That keeps the move list short enough for a model to actually work through in its scratchpad, while still leaving plenty of room for 3- to 5-ply traps.

Plus, I trained a few Monte Carlo Tree Search and AlphaZero engines in Rust (ai-crosshair) to set a baseline for engine strength. The neural network is only 114K parameters and trained in six minutes on my desktop GPU, which makes it a fun yardstick for frontier language models: