animate-transcript.mp4
This is a simple project to generate animated transcript from an audio recording. You can use a screen recording tool to record the animation to have a video like Descript's.
This project doesn't generate a video automatically. You can use a screen recording tool of your choice to record the animation. Refer to the demo video to see the basic functionality.
This project depends on gentle, a tool for aligning speech with text, running inside Docker. You will need to have Docker installed in your system to run this project.
# start gentle service, running as a Docker container
pnpm gentle:start
# start a cors proxy and the next.js app
pnpm devOpen http://localhost:3000 with Google Chrome and start recording!
If you are done with the app, you can stop the gentle container with the following command:
pnpm gentle:stopAs far as I know, the SpeechRecognition API really only works reliable on Chrome for Desktop. So this project won't work well on Safari, or even Chrome on Androids.
It is also currently limited to only the English language.
No, at least not at the time. If you'd like to do it, it should be pretty easy to do so though. The Next.js app should be able to be hosted on Vercel for free. Then it's just a matter of finding a cloud provider of your choice to run the gentle Docker container and the cors proxy.
- The Next.js app uses the MediaRecorder and SpeechRecognition APIs to create an audio recording, along with the text transcripts.
- The audio file and the text transcript is sent to a locally running
gentleservice.gentlewill send back a JSON containing recognized words along with the timestamp it was spoken at in the recording. - The JSON, along with the audio recording is used by the
<Player />component to animate the transcript.
If you would like to customize the stylings and make it more personalized to your brand, take a look at the src/components/Player directory.