qmmr / fifa-worldcup-2018-scraping

Set of tools to scrape fifa.com/worldcup, collect and store worldcup 2018 data in your MongoDB database

FIFA Worldcup 2018 data scraping

Requirements:

• node.js >= 10.0
• mongodb database

Usage:

• clone the [e-mail]:qmmr/fifa-worldcup-2018-scraping.git
• cd fifa-worldcup-2018-scraping
• create .env file to set env variables needed to connect to your chosen MongoDB server
  • dotenv
  • If you want to use your local development server you can use these.

    DB_USER=admin
    DB_PASSWORD=password
    DB_HOST=127.0.0.1
    DB_PORT=27017
    DB_NAME=fifa-worldcup-2018
    

• make the files executable chmod u+x ./insert-teams.mjs ./insert-games.mjs ./update-teams.mjs ./insert-lineups.js ./insert-groups.js
• run the commands in this order:
  1. node -r esm src/insert-teams.js
  2. node -r esm src/insert-games.js
  3. node -r esm src/update-teams.js
  4. node -r esm src/insert-lineups.js
  5. node -r esm src/insert-groups.js

1. Insert teams populates your DB with teams data from teams.json.
2. Insert games scrapes the https://www.fifa.com/worldcup/matches/ and populates your DB with games data.
3. Update teams associates the games with teams (in order to fetch teams games).
4. Insert lineups scrapes the match lineups from fifa.com/worldcup/matches/match (example match)
5. Insert groups insert information about groups and results of games

Have fun! 👋