dfreelon / pyktok

A simple module to collect video, text, and metadata from Tiktok.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add Possibility to return larger comment threads

pkreissel opened this issue · comments

Recently I reverse-engeneered the API used by TikTok to download comment threads, and it is actually pretty simple. You just need the video ID and a "cursor" to paginate the results. I could try a PR if you think it might be a fit for this module, maybe with an optional param to try and save the entire thread?

Example:

def video_comments(video_url, video_id):
        """Retrieve and store Tiktok Comments for a video"""
        import requests
        hasmore = True
        cursor = 0
        headers["referer"] = video_url
        while hasmore:
            params = {
                'aid': '1988',
                'app_language': 'ja-JP',
                'app_name': 'tiktok_web',
                'aweme_id': video_id,
                'browser_language': 'de-DE',
                'browser_name': 'Mozilla',
                'browser_online': 'true',
                'browser_platform': 'MacIntel',
                'browser_version': userAgent,
                'channel': 'tiktok_web',
                'cookie_enabled': 'true',
                'count': '20',
                'current_region': 'JP',
                'cursor': str(cursor),
                'device_id': xxxx,
                #....need more experiments to see what of these params is necessary here.
            }

            try:
                response = requests.get(
                    'https://www.tiktok.com/api/comment/list/',
                    headers=headers,
                    params=params,
                    cookies=cookies
                )
                data = response.json()
                print(data["has_more"])
                cursor = cursor + 20
                if (data["has_more"] != 1) or (len(data["comments"]) < 20):
                    hasmore = False
                for comment in data["comments"]:
                    savecomment_to_file(comment)
                time.sleep(random.randint(1, 4))
            except Exception as e:
                print(e)
                hasmore = False

Hi, thanks! I added an edited version of your function to Pyktok as save_video_comments. I credited you in the README but happy to offer additional credit elsewhere if you like, just let me know.