I wrote the experiment code, generated 15,000 teacher-labelled positions, trained nine models, and analyzed the results. Professor Guerzhoy provided theoretical guidance and helped define the experimental direction.
Outcome
None of the eight convolutional students beat the linear baseline on held-out positions in this experiment. The baseline’s mean absolute error was 205.10 centipawns.
Can a small model learn how a chess engine sees the board?
My undergraduate thesis began with a broad question: why did computers surpass the strongest human chess players years before achieving the same milestone in Go? Search-tree size explains part of the difference, but not how difficult a position is to evaluate without deep search.
I narrowed the project to approximating a chess engine’s position evaluations. The planned Go comparison was not completed within the project timeline.
02 · Experiment
Turning the question into a teacher–student system.
Modern neural-network chess engines combine position evaluation with tree search. My experiment isolated the evaluation problem by treating Leela Chess Zero, or LC0, as a teacher model. Instead of asking a smaller model to select moves or play complete games, I trained it to predict LC0’s numerical evaluation of a board position.
This is a form of knowledge distillation: a capable model produces targets that smaller models attempt to approximate. Here, the targets came from LC0’s search-based analysis, rather than its neural-network output alone.
Knowledge distillation
One position. Two paths.
The teacher supplies a target evaluation. The compact student learns to approximate it from the same position.
01 · Source
Chess position
rnbqkbnr/ pppppppp/ 8/8/8/8/ PPPPPPPP/ RNBQKBNR
FEN stores a complete board state in one compact string.
Teacher path02A · Label
LC0 evaluates
Ten seconds of engine analysis produces the target score.
+84 centipawnsStudent path02B · Learn
8 × 8 × 7 tensor
Piece type, colour, and location pass through a compact model.
+71 centipawns
03 · Compare
L1 error
The absolute difference trains the student.
|84 − 71|13 cp
Repeated across 15,000 positions and nine student architectures. Scores shown here are illustrative.
03 · Data pipeline
15,000 teacher-labelled positions.
I sourced public chess positions represented in Forsyth–Edwards Notation, configured LC0 locally, and evaluated each position programmatically. LC0 received a ten-second evaluation window and returned a score in centipawns, where 100 centipawns corresponds approximately to one pawn.
15,000positions evaluated
10 secper teacher evaluation
~42 hevaluation budget
The resulting workflow read each position, requested an LC0 score from White’s perspective, converted the board into a tensor, and stored the input with its teacher target for training.
04 · Representation
Seven channels for piece placement.
I tested a 12-channel representation with a separate channel for every piece-and-colour combination and a smaller seven-channel representation with six piece-type channels plus one colour channel. Neither showed a meaningful generalization advantage, so I retained the smaller 8 × 8 × 7 tensor.
The encoding captured piece type, colour, and location. It omitted side to move, castling rights, en passant state, move counters, and game history, so it did not represent the complete game state. Higher-level patterns such as king safety and pawn structure had to be learned from the spatial input.
Representing a chess position
From board state to model input.
The compact representation separates piece type from colour. Choose a channel to see exactly which squares become active.
Example FENr1b1k1nr/pppp1ppp/8/8/2B1P1nq/3P2P1/PPP2b1P/RNB1K2R w KQkq - 0 8
Choose one of seven channels
3 active squares
Original position
White to move
Pawn channel
8 × 8 binary matrix
Channel 1 · Pawn
A value of 1 marks every pawn, regardless of colour. Piece colour is stored separately in channel 7.
Derived from the original thesis implementation. In the colour channel, empty squares and black-occupied squares both remain 0.
05 · Student architectures
Nine models with increasing capacity.
The comparison included one linear baseline, four Conv2D configurations, and four Conv3D configurations. The linear model assigned additive weights to the piece-type and colour channels at each square; the convolutional models were intended to capture more complex spatial relationships.
I trained on approximately 12,000 positions and evaluated against 3,000 held-out positions using L1 loss, interpreted as mean absolute error in centipawns.
06 · Held-out performance
More capacity. No better generalization.
The convolutional students learned the training examples more closely, but none produced a lower held-out mean absolute error than the linear baseline.
Central findingZero of eight convolutional models beat the simplest model.
Best held-out MAE205.10centipawns · linear baseline
Held-out mean absolute error by architecture
Approximately 3,000 held-out positions · lower error is better
← Lower is better
Architecture200210220230235MAE · cp
LinearBaseline
205.10
Conv2D0 hidden layers
219.55
Conv2D1 hidden layer
232.96
Conv2D2 hidden layers
219.67
Conv2D3 hidden layers
215.06
Conv3D0 hidden layers
206.73
Conv3D1 hidden layer
210.03
Conv3D2 hidden layers
215.16
Conv3D3 hidden layers
209.60
LinearConv2DConv3D
Linear baseline: 205.10 centipawns. Conv2D with zero, one, two, and three hidden layers: 219.55, 232.96, 219.67, and 215.06 centipawns. Conv3D with zero, one, two, and three hidden layers: 206.73, 210.03, 215.16, and 209.60 centipawns.
Training data
Deeper models fit more closely.
Training loss generally decreased as layers were added, indicating greater capacity to fit observed patterns.
→Held-out positions
The advantage disappeared.
Held-out error fluctuated rather than improving with depth, exposing the experiment’s generalization gap.
Metric: L1 loss, reported as mean absolute error in centipawns. Values are reproduced from the thesis results tables.
07 · Limitations & next steps
What this experiment can tell us.
The negative result applies to this dataset, encoding, training procedure, and family of shallow architectures. It does not establish that compact chess evaluation is impossible. Predicting a teacher’s centipawn score measures similarity to that teacher, not playing strength.
The experiment was constrained by dataset size, compute, model selection, and the way positions were divided. The split used held-out positions rather than a verified game-level separation, so it does not establish generalization to entirely unseen games.
What I would change today
I would begin with a substantially larger dataset and a reproducible game-level split. The source file stored consecutive positions from games, so complete games should be assigned between training, validation, and testing with a fixed seed. Multiple seeds would show how stable each result was.
I would add constant and material-based baselines, expand the architecture search, examine the teacher-score distribution more carefully, and measure both prediction error and playing strength by placing each learned evaluator inside a search procedure.