🇯🇵🇺🇸🇫🇷 Tranzlate: The Zoom Translator 🇨🇳🇪🇸🇩🇪

Click image below to watch demo video:

Currently, many classes, meetings, talks, and academic conferences (CVPR and PLDI this summer are two I'm aware of) around the world are being conducted online via Zoom due to the COVID-19 pandemic. Previously, people around the world would have had to travel for many of these events, but they now have access to these resources due to their virtual nature. A major barrier, however, is language. For instance, a person from the U.S. may have always wanted to attend an annual French conference. Yet, even though it's now conducted on Zoom, that person still cannot attend because he/she does not understand French. This is where Tranzlate comes in. The user interface is minimalistic, with a Zoom meeting screen on top and the translated text on the bottom. It is also easy to use: simply enter your meeting ID, select the speaker's language and the language you'd like to translate to, and you are all set.

Prerequisites

This app uses the IBM Watson Speech to Text API and the Google Translate API, so you would need these appropriate API keys. Optionally, you may also install a virtual audio device software.

IBM Cloud

Sign up for an IBM Cloud account
Download the IBM Cloud CLI
Create an instance of the Speech to Text service and get your credentials:
- Go to the Speech to Text page in the IBM Cloud Catalog
- Log in to your IBM Cloud account
- Click Create
- Click Show to view the service credentials
- Copy the apikey and url values

Google Cloud

Sign up for a Google Cloud account
Create a project via the Google Cloud console
Enable the Translation API in your project and create an API key

Virtual Audio Device (optional, see "Using the application" below)

Windows and Mac OS: VB-Cable
Linux: PulseAudio Volume Control and Audacity

Entering your credentials

In the project directory, copy the .env.example file and create a file called .env
```
cp .env.example .env
```

Open the .env file and add the service credentials that you obtained in the previous step

Example .env file:

SPEECH_TO_TEXT_IAM_APIKEY=X4rbi8vwZmKpXfowaS3GAsA7vdy17Qh7km5D6EzKLHL2
SPEECH_TO_TEXT_URL=https://gateway-wdc.watsonplatform.net/speech-to-text/api
GOOGLE_TRANSLATE_APIKEY=X4rbi8vwZmKpXfowaS3GAsA7vdy17Qh7km5D6EzKLHL2

Running locally

Install the dependencies
```
npm install
```
Run the application
```
npm start
```
View the application in a browser at localhost:3000

Using the application

Enter your Zoom meeting ID upon first loading the website. You will then see your Zoom meeting window with a text box below, where the text translation will be displayed. Below the text box, select the speaker's language on the left and select the language you want to translate to on the right. When you want to start translating, click the "Start Transcribing / Translating" button and voilà!

Important: As Zoom's SDK does not provide the capability to obtain the meeting's live audio stream, this app transcribes and translates the meeting based on your microphone recording; thus, you need some way of playing the meeting audio through your microphone. The easiest way to accomplish this without any external software would be to connect your audio output to an external speaker, which your microphone can then record. Otherwise, you would need to follow some simple additional steps. If you are on Windows or Mac OS, use VB-Cable; if you are on Linux, follow these instructions.

Acknowledgments

This project uses IBM's Speech to Text demo as starter code. All modifications are clearly denoted Modified or Created at the top of relevant files, in compliance with the Apache-2.0 License.

Thank you to the staff of HackThis for organizing this great event; I had a lot of fun working on this project!

remyang55 / tranzlate