Cross Online social Network Crawler to link users from Twitter, Instagram and Foursquare.
find_users.py searches through Twitter for users who crossposted Instagram and Foursquare content and outputs a csv file with the user ids on Instagram and Twitter. The Foursquare ID is not given because the Foursquare API doesn't let you fetch users, but the users in the list have posted content from Foursquare.
fetch_data.py provides routines to fetch profile and user media when given user ids from instagram and twitter, and ids from media from twitter, instagram and foursquare
read_data.py routines that read data of the format given in 'sample_data', fetches data and dumps a pickle of the user
Best way is to get the code running quickly using virtualenv: http://docs.python-guide.org/en/latest/dev/virtualenvs/
Then run:
pip install -r requirements.txt
You need to fill / create the files instagram_config.py, foursquare_config.py and twitter_config.py and put the details (app id, app secret) of the apps you create at the respective platforms.
To fetch data, you can test:
python read_data.py sample_data/users_linked.csv sample_data/user_tweets.csv sample_data/user_instaphotos.csv sample_data/user_4sqcheckins.csv
The full dataset is here: http://cake.da.inf.ethz.ch/OSN-sigir2016/
To get a snapshot of the dataset from February 2016 that you can create with these scripts, get in touch by mail with han.veiga@gmail.com