Paper Shows Arena-Style AI Leaderboards Can Be Manipulated by Up to 45 Ranks

Researchers published an arXiv paper on Nov. 10, 2026, showing that arena-style rankings for language models can be manipulated through strategic voting, strategic prompting and strategic nomination. The study examined public leaderboard data and found that providers could boost a model’s rank by inserting irrelevant models, choosing favorable prompts, or casting votes that distort the Bradley-Terry aggregation used to produce final rankings. The paper describes safeguards against each attack.

The paper’s central measurement is rank movement. It says the attacks boosted the rank of nearly every tested model, with each attack capable of lifting a model by 15 to 20 places. When combined, the strategies could move a model by as much as 45 places. The authors say the vulnerability applies to any preference-based evaluation using Bradley-Terry rankings that allows open prompting, voting, or model submission.

Arena leaderboards became popular because they compare anonymous models on real user prompts and aggregate pairwise votes, aiming to capture preferences that static tests miss. The paper’s finding matters because those rankings appear in research papers, production evaluations and public comparisons. If open voting, prompting or submission can be exploited, a high score no longer proves broad superiority; it may reflect control of the evaluation path. The study does not show a specific provider manipulated a named leaderboard, but it shows the mechanism is feasible.

The result pressures developers, researchers and investors who use leaderboard positions to choose models or assess progress. A 45-place jump can make a weaker model look competitive on a crowded board, while a stronger model can be pushed down by irrelevant entries or biased votes. The paper says safeguards can be built against each attack, but does not report a deployed fix. Until controls are in place, leaderboard rank alone is weak evidence.