This project was developed within the scope of independent studies at the University of Trier. The main purpose was to get an overview of current machine learning techniques and especially Convolutional Neural Networks.
ChessML performs basic board localization and piece recognition from photographs taken from the above a chessboard. This is done using OpenCV for image processing and PyTorch as framework for the neural network. The training/validation data was generated and labeled by myself and is available here. The trained model is available here.
The localization of the chessboard is based on the same core idea as in the tutorial by Nesh Patel [1] towards the localization of sudoku fields with a few adaptions and alterations. The process breaks down to 9 parts:
- Grayscaling the image
- Blurring the image
- Edge detection using Canny edge detector
- Dilate to get thicker lines
- Hough transform to identify horziontal and vertical lines
- Calculation of the intersections of the lines
- Clustering of the interesections
- Identifying the corners of the chessboard
- Warping the image
After this, the image is cut into 64 tiles which are then saved in an output directory:
...
The data for training the neural network is based on 417 photographs of different arrangements on a single chessboard which resulted in 3900 training images (300 per chess piece + empty fields) and 7.800 validation images. Each image was labeled using a simple labeling tool which was specially designed for labeling the chess pieces. It functions by simply moving every image to a corresponding folder named for the pieces they contain. Afterwards, the data was augmented using this script. The data augmentation process resulted in 31.200 training images and 7.800 validation images.
To get a better understanding on how convolutional neural networks work, the first version was built as a completely new CNN which was trained on Google Colab. The results are shown below:
Chess pieces are labeled using their common notation:
bb = Black Bishop
bk = Black King
bn = Black Knight
bp = Black Pawn
bq = Black Queen
br = Black Rook
...
Epochs: 50
Best-Accuracy after epoch: 35
Best-Accuracy: 62.56%
| Piece | White | Black |
|---|---|---|
| Bishop | 74 % | 24 % |
| King | 77 % | 38 % |
| Knight | 85 % | 83 % |
| Pawn | 53 % | 59 % |
| Queen | 35 % | 66 % |
| Rook | 63 % | 51 % |
| Empty | 100 % |
Since the result of the first training had a very poor accuracy, the training was done again using a pretrained version of resnet18 from PyTorch. The result was pretty good and it ended up with an accuracy of 99%.
Epochs: 20
Best-Accuracy after epoch: 12
Best-Accuracy: 99.00%
| Piece | White | Black |
|---|---|---|
| Pawn | 99 % | 100 % |
| Queen | 98 % | 96 % |
| King | 98 % | 99 % |
| Bishop | 98 % | 96 % |
| Knight | 100 % | 99 % |
| Rook | 100 % | 100 % |
| Empty | 100 % |
- Finish README.md
- Improvement of the board localisation (error handling etc.)
- More data for different type of chessboards and pieces
- Generation of FEN
- App for analyzing board situations
- Recommendation for the next move
When the idea for the project came up, it was almost certainly clear that somebody out there must have had the same idea. Many parts of the board localization are based on [1, 2].
[1] https://medium.com/@daylenyang/building-chess-id-99afa57326cd
[2] https://medium.com/@neshpatel/solving-sudoku-part-ii-9a7019d196a2
To my wonderful friend Aaron Winziers for making linguistic contributions to the project and its documentation