YuhanXiaoJY / Efficient-crawler-for-Twitch-videos-and-chatting-records

An Efficient Crawler for Twitch Videos and Chatting Records

Goal

• Efficiently crawl videos and relevant chatting records from Twitch
• Convert the video to the format of 3fps
• The code just crawls videos and chatting records concerning NBA and GTA live broadcast. You can change the url in sinVidCom.py and videolist to crawl other data.

Guide

• Run the code by the command python2 main.py in the ubuntu environment
• The code needs ffmpeg and youtube-dl to conduct the work. Please make sure you have the libs installed
• Videos are saved in Video folder
• Chatting records are saved in ansNBA.json and ansGTA.json in the Data folder. See the format of json at jsonformat

Author

Yuhan Xiao