MichaelKreil / twitter-analysis2

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

twitter-analysis

I have been using this framework for 5 years (since March 2018) to scrape and analyse numerous topics on Twitter, including: Corona, George Floyd, Trump, US election 2018, Ukraine, climate change, ...

Since 12/4/2023, the API has been closed due to Elon Musk's restrictive API policy.

But I managed to collect 23 billion tweets (120 TB uncompressed, 5 TB compressed). Each tweet is archived as API JSON response, so no images or videos.

Here is an example docs/georgefloyd6_2020-06-02-extract.jsonstream.xz (The first 10,000 tweets of the day. The entire day comprises 32 million tweets, 177 GB uncompressed, 9.7 GB compressed.)

A list of all queries can be found here: docs/queries.tsv

I am open to suggestions on how to make these 5 TB available for download.

About


Languages

Language:JavaScript 96.7%Language:CSS 1.5%Language:HTML 0.9%Language:Rust 0.8%Language:Shell 0.1%