The ultimate YouTube Comments Analyser Tool, presented as a NLP project.
| Topics | Jump to Link |
|---|---|
| Why I built this app | because the thought matters |
| How Insight works | so that you love it as much I do |
| Watch the demo video I made for the app | this was hard to make but I enjoyed it very much |
| Contribute to this app (build a frontend) | yes, please. You're welcome to. |
| Run the app on your machine | so you get a taste |
| How I built this app (included with tutorials) | tools I used to make it happen |
| Your support matters! | I would appreciate it very much |
The idea for this project was birthed from a tweet I saw about a famous YouTube creator saying that a tool that can help analyse the enormous amount of comments that they get on each video wasn't yet present on the market. Being a data science practitioner and an avid YouTube fan, that tweet picked my brain and I was like, this should be fun little challenge. 😸
From then on, what began as an idea of a good, practical data science project, slowly developed into a full fledged web application with the help of the awesome frontend libary for data scientists called Streamlit (more later on this).
Insight currently offers these features:
- Fetch all comments from any video from a YouTube video by pasting a URL, or
- Select and load any pre-existing csv file of comments that you might have already scraped before
- Perform the following operations on the comments:
- Cluster the set of comments into different topics with a percentage assigned to each topic pertaining of how much people have talked about that something in the comments
- Find top used emojis by the commentors 😍
- Write a query and get comments semantically related to that query available to analyse
- Form a pretty little word cloud of the comments, using any of the shapes in the drop down list like a cloud(duh), a dog, a retro camera, and some others!
This is what the app looks like:
You can also watch the demo video I made about the project (It's 3:16 long. I worked very hard on this to make it fun rather than just a boring demo):
First of all, please star this repo if you like it. ⭐ Thanks! Now let's move on.
I made Insight with 2️⃣ things in mind:
- I wanted to build a real, usable app like this with inspiration from some of the awesome YouTubers like Peter Mckinnon that I love. The comments from his videos are actually being used in the demo video I made to showcase the project. Go watch it 👆🏻.
- I wanted to use and hone my skills in NLP by demonstrating its power to the world wth a practical app like this.
I have always been a backend-Java-Kotlin-Python guy and thus, I am unfortunately, quite lacking in frontend web development skills. Therefore, if you are someone who is excited by the immense possibilites associated with this project and wants to make a modern, simplistic frontend for it, PLEASE ping me in here via a new Issue or contact/DM me on Twitter or LinkedIn whenever you can.
I would love to collab with someone like you who is eager to use her/his skills make this project more awesome! Thanks for being here anyway. ❤️
- Make a new directory to store the project:
mkdir InsightApp- cd into the directory:
cd InsightApp- Clone this repo:
git clone https://github.com/yashprakash13/Insight.git- Make a new virtual environment. I recommend using pipenv:
pipenv shell- cd into the app directory:
cd Insight- Build the app with docker:
docker build --tag insight:1.0 .- Run the app:
docker run --publish 8501:8501 -it insight:1.0- Navigate to https://localhost:8501 to explore the app! Don't forget to see the demo video first to make sure you have a good grasp of the app's workings!
If you want to build your own comments dataset with this app, feel free to follow the entire process I described in this blog post. This will help you make your own creds.json file that you can keep in the root project directory when you run the app.
I'm also making an online course-ebook with the WHOLE guide to building this project. You can book your copy here. It's free too. :)
Insight was built with these major components. A huge thank you to their creators for making these amazing tools available for free.
- Google YouTube API -> to download comments from videos, as I've described here.
- Sentence Transformers -> to enable semantic similarity search and information retrieval with by computing sentence embeddings, as described here.
- Streamlit -> the easy frontend development library for backend people like me :P I wrote a detailed blog post around it too here.
- Docker -> to easy deploy the app as a stand-alone container, like I've written here.
You might have a good idea by now that I absolutely LOVE to write. 😁
Did you like this project and the features that it enables or are you hyped by the awesome use of NLP demonstrated in here? Please consider supporting me! :)
Buy me a ☕ or a 🍺
