duyuankai1992 / Paper-Piano

Piano like no other, Piano on Paper

PAPER PIANO

Now we don't need to buy Piano if we want to play music. We can play piano on paper, although it may not give feeling of pressing keys on piano but it gets work done!

demo.mp4

Currently it only supports maximum 2 fingers (1 finger of both hands). Support for more fingers and highly susceptible training model is on the way.

1. Setting up project
2. How to Use
3. Training and Adjusting
4. Contributing to this project

SETTING UP PROJECT

Python version 3.11 and above

1. Clone the repository git clone https://github.com/Mayuresh1611/Paper-Piano.git
2. run command pip install -r requirements.txt in the command line.
3. Execute run.py file

HOW TO USE

This is a little trickier part as the project requires you to set up a webcam in the specific angle at a specific height and distance. Also stronger the light, better the performance.

STUFF YOU WILL REQUIRE

1. webcam or you can use third-party tools for webcam.
2. Two A4-sized white paper, horizontally joined together. 2 rectangles need to be drawn at both ends of the paper with a black marker, thicker lines yield better results.
3. Recommended position for the webcam will be such that it can capture the finger and shadow beneath the finger and should have both boxes we drew on joined paper in the FOV of the camera. Just like shown in the demo video.
4. Light source in front, ie. behind the camera would be preferred. Casting sharp shadows.
5. Hand with all fingers.

TRAINING AND ADJUSTING

• During training the model on your finger, the first window will appear where box will be drawn around the tip of the finger. If the box does not cover complete finger and little surrounding of the finger. Adjust the camera accordingly.
• In the training stage, do not move your finger furiously, move it slowly giving out every angle.
• While training, during the touched finger state, do press the finger but not too hard. In an untouched finger state, do not touch paper, you can get near paper but not too close. Lift the finger high like you would normally do.
• CNN has been used to train over the data, it distinguishes the touched and untouched finger, if the results are not satisfactory, retrain the model.

CONTRIBUTING TO THIS PROJECT

If we make this project into a complete working piano on paper. It would give access of piano and instruments for those who cannot afford it. like me :)

• There are no set rules or code of conduct as of now. Anyone wtih better ideas and improvement is welcomed.
• The only rules are

1. The name of the function should match the functionality it is doing.
2. Use comments only when required.
3. Share with the ones who can improve this project even more.