yoogottamk / visudoku

Solves a sudoku puzzle by looking at it

ViSudoku

This is a visual sudoku solver. It takes a picture of a sudoku puzzle as input and then solves and prints the solution.

The whole opencv workflow is explained in visudoku.ipynb.

Training data (570 images) was generated by hand. Then, it was augmented in the following ways:

• added black border of {1..4}px around the image to simulate smaller digits
• removed black border of {1..4}px from the image to simulate larger digits
• rotated {4, 8, 12, 16} degrees clockwise and anticlockwise to simulate weird orientations

Digit recognition is done using K-Nearest Neighbours (knn).

The final puzzle is solved by a cpp program (here as a submodule) which I wrote a few years ago.

UPDATE!

Now I have 1332 images, and a better model. The augmentation was limited to {1..2}px scaling and {4, 8} degrees rotation.

It performs wayy better now!

Screenshots

An example

Instructions

Docker

Make sure you have docker installed!

You can either build the image yourself (takes a lot of time) or use the online one

Build locally

To get started, run

./setup.sh
docker run -p <port>:8080 -d --name visudoku visudoku 

Download built image

Simply run

docker run -p <port>:8080 -d --name visudoku yoogottamk/visudoku

You can check the app at the port you specified

To stop/start the container, run

docker stop/start visudoku

Normal

Before anything else, please create a python3 virtual environment and install the stuff in requirements.txt

python3 -m venv venv
. venv/bin/activate

pip install -r requirements.txt

Please first follow the instructions given in digit-recog

Flask

./flask-setup.sh
cd flask-app
python server.py

You can check the app at port 8080

Jupyter

NOTE: You first need to install jupyter notebook/lab first

Run jupyter notebook and open visudoku.ipynb.

You can either use the images given in test-images, or provide your own images, or even use the webcam. To use the webcam, you have to provide an invalid image pat.

If you use the webcam, you have to press c when you want to capture the image.

After getting the image, it will automatically solve and give the solution at the bottom.

Aside

This project began a year ago, I could only work on it in my vacations.

winter'18

• Decided that I wanted to make something like this
• Finished the opencv part (everything till digit extraction)
• Got stuck at digit recognition.

summer'19

• Took a course on ml
• Tried to make it recognize the digits, failed terribly
• Realised that digit extraction is really bad, rewrote it
• Wrote a solarized-dark theme usercss for jupyter notebook and jupyterlab

winter'19

• Tried digit recognition again, using a CNN, accuracy: 9%
• Tried digit recognition using knn, accuracy: 99.69%
• Put up all the pieces together

TODO: organize this in a better manner, create a better frontend, and tidy up the code