Independent Analysis of AI Agents Playing Stronghold Crusader

The score is 100 × (0.8 × net worth growth + 0.1 × population growth + 0.1 × popularity ÷ 100). Growth is measured on each map from doing nothing, 0, to the person’s game, 1. Population growth and popularity stop at the person’s level; net worth growth can run from −1 to 2. A map’s score is the mean of a model’s full-budget episodes on it, one game with weapons and armour counted in net worth and one without, and the score over every map is the mean of its map scores.

Net worth is gold plus every stored good valued at the game’s fixed marketplace sell price: wood 1, stone 7, iron 23, bread, cheese, meat and apples 4, wheat and hops 8, flour, ale and pitch 10, weapons and armour 10 to 30 (nothing in the “no arms” game). Scoring in full

Selling at the marketplace converts goods to gold at the price the score already uses, so it does not change net worth. It does free up the stockpile, and production stops when the stockpile is full.

Population and popularity make up 20% of the score, 10% each; population is measured as growth from doing nothing towards the person’s game. Worker efficiency rises with fear, from gallows, gibbets and the game’s other “bad things”, and falls with “good things” such as gardens.

The game is paused while the model thinks, so token use does not affect how much game time it gets. Each turn that runs the game lasts at least 8 game seconds, which limits the number of requests in a game and therefore its cost. A cost is what OpenRouter billed; where an API reports tokens but no bill, such as Anthropic’s and Moonshot’s, it is the game’s tokens at the model’s list prices, marked ~. The interview after an episode is not in the game’s cost.