yxgeee / RobotCarDataset-Scraper

Downloads files matching patterns from the Oxford Robotcar Dataset website

Introduction

This script can be used to perform sequenced downloads of various sensors logs for the robotcar-dataset https://robotcar-dataset.robots.ox.ac.uk/.

The code is tested for Python3 on Ubuntu 16.04.

Getting started

wget https://raw.githubusercontent.com/yxgeee/RobotCarDataset-Scraper/master/Dockerfile
docker build -t robotcar-dataset-scraper .

Example usage

Mount your downloads dir through the docker image:

docker run --rm -it -w /RobotCarDataset-Scraper -v $HOME/Downloads:/Downloads robotcar-dataset-scraper:latest

The script (optional!!)

python get_datasets.py

parses the html for the complete dataset listing and scrapes each dataset page and retrieves the available files (tar) for download. With the example output file datasets.csv provided, you should then run:

python scrape_mrgdatashare.py --downloads_dir /Downloads --datasets_file datasets.csv --username USERNAME --password PASSWORD