Giters

gwu-libraries / social-feed-manager

"Old SFM" -- manage rules and streams from social data sources, starting with twitter.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

fetch_urls errors

edsu opened this issue · comments

commented

I see a lot of messages like this when running the fetch_urls management command:

Error fetching http://t.co…: ('Connection aborted.', gaierror(-2, 'Name or service not known'))

It looks like somehow the abbreviated url (as it appears in the text of the tweet) is being resolved rather than the valid url that should be available in the tweet metadata?

commented

Twitter embeds the original tweet as retweet status object. These urls are occurring in case if Retweet's with length greater than 140 characters. In such cases twitter truncates the long text and adds an ellipsis “…”. In such cases if the text doesn't have any other url then these ellipsis “…” are converted to 'http://t.co…' urls when fetch_urls management command is fetching the urls from text as there is no url available in the tweet metadata.

Please refer to following link for explanation :
https://dev.twitter.com/overview/api/entities-in-twitter-objects#urls

Since these are not actual urls i am checking for these conditions and will update the code soon for it to ignore fetching these from Retweet's text.