An open source platform to annotate and label data at scale
Shoonya is an open source platform to annotate and label data at scale, built with a vision to enhance digital presence of under-represented languages in India.
Shoonya offers support for multiple data types (Ex : parallel datasets, OCR, ASR, TTS etc) and labeling tasks (Ex : parallel datasets, OCR, ASR, TTS etc).
Shoonya, referring to zero, represents the start. It is also represents universality in the sense that several cultures have a similar symbol for zero. We believe that the language resources that we collect in Shoonya will start a valuable open-source movement for Indian language technologies that will be available universally for all to adopt and improve upon.
The National Language Translation Mission (NLTM) has been announced in the budget by the Honorable Finance Minister in the backdrop of growing demand for accessing online services in local Indian languages. This will enable the wealth of governance-and-policy related knowledge on the Internet being made available in major Indian languages. The Ministry of Electronics and Information Technology (MeitY) has launched 'Bhashini' to help ensure that digital content is readily available to all citizens, in their preferred languages.
The goal of Bhashini is to develop an ecosystem of innovative practices for data collection, curation, develop technology for speech to speech translation and deliver solutions powered by open data, apps and services. Bhashini shall act as an orchestrator to bring contributions (like data, models etc.) received from government, industry, academia and society into an open “Hundi” or “Repository”. All contributions to Bhashini shall be validated and standardized using a Unified Language Contribution API (ULCA).
Reference : Bhashini Whitepaper
Also read Bhashini Data Report
Naturally, data collection/curation becomes the core of building state-of-the-art NLP ML models. This is where Shoonya comes into picture. Shoonya provides the platform for the Annotators/Translators to create such large datasets with highest quality.
- Support all possible data types and labeling tasks
- Build a reliable & scalable platform beneath Shoonya
- Keep the UI simple and intuitive
git clone --recurse-submodules https://github.com/AI4Bharat/Shoonya
Any information/help/discussion required, can be taken up using the following link : https://github.com/AI4Bharat/Shoonya/discussions