Download tweet threads from first tweet (not just last)
clintgibler opened this issue · comments
It would be convenient if I could download a tweet thread by supplying a link to the first tweet, rather than the last one.
I assume there's a Twitter API reason why you've done this behavior, perhaps because it's not easy to programmatically determine the "end" of a thread, or maybe other people's tweets get mixed in. I'm not sure, I haven't used the Twitter API to gather whole threads before, just metadata about individual tweets mostly.
Anywho, if this is easy, it could be useful, but if it's not, may not be worth doing it, as navigating to the end of the tweet thread isn't too much work.
Yes, this is a very bothersome bit about the Twitter API. Each thread is assigned a Conversation ID. You can only search by conversation ID with one of two endpoints:
- recent search (limited to the last 7 days of tweets)
- full search (only available to academic research plans)
So to get a thread, I ask for the last tweet because that tweet has the ID of the tweet it's responding to. So I can work tweet by tweet back up the thread until I hit the top (the tweet that isn't in reply to anything).
Pros:
- We can get threads even without an academic license and aren't limited to only new tweets
- There's the added side effect of being able to capture replies/conversations easily. If you grab a tweet that's replying to a thread, or the last tweet in a line of replies, it'll work back up the chain and grab them all.
Cons:
- It's inconvenient to have to find the last tweet of the thread when you only want the thread itself.
- We have to make a separate network request for each tweet in the thread instead of grabbing them all at once.
If Twitter ever opens up their conversation search to more than just the academic level, I'd happily revisit this.
Ah makes sense, I figured it was something like this, thanks for sharing the context!