francisc-czobor / cars2click-test

cars2click test app for the recruitment process

cars2click-test

This app was written during the cars2click recruitment process.

Its purpose is to scrape the car dealership data from this website. The output is a JSON file containing a list of data points with the following format:

  {
    "brand": "Car Brand",
    "dealer": "Car Dealership",
    "address": "The address",
    "tel": "0123465789"
  }

Installation

You need Python 3.6+ and the Scrapy Python library. I recommend using a virtual environment.

To create a virtual environment run python3 -m venv venv. Then you need to activate it by running source venv/bin/activate.

Install all the required components (in the activated virtual environment) by running python -m pip install -r requirements.txt from the root of the project.

For more information check the following links:

• Virtual Environments
• Scrapy Installation Guide

Run the scraper

To run the scraper you have to run the following command from the concession directory:

scrapy crawl concession -O results.json

This command will create a JSON file called results.json in the concession directory which contains the scraped data. From my testing the crawler manages to scrape 4882 data points.