Ankit1598 / sound-mixer

Hack At Home II HACKATHON PROJECT SUBMISSION

Inspiration

Ever wanted to lead your own choir? Want to be a sound mixer? Wanted to generate music just by your moving your hands? Its not some magic! !

What it does

It determines what hand gesture you are making from the American Sign Language and plays music corresponding to each of them.

How we built it

• First, we made a webapp using React js with a cool Music Visualizer.
• Developed a Deep Learning model using Cognitive Services.
• Integrated the model with webapp and made an API for the same.
• Added the Keyboard Music Player.

Challenges we ran into

• Experienced great difficult in sending the image from the frontend to the API as API accepted only URLs

Accomplishments that we're proud of

• Even though in the 24hr hackathon, we made an awesome interface with an authentication system.
• Developed a Highly accurate Deep Learning model and trained over a data of thousands of images.
• We managed the work very well right from the team formation to the providing the MVP.

What we learned

• About postman API.
• Conversion of Image to DataURL and vice versa

What's next for Sound Mixer

• At moment, it works on images(kept due to less number of Cloud credits). When brought commercially with huge number of cloud credits, can be expanded to videos as well.