- β Introduction
- π― Laravel project requirements
- βοΈ Installation/Configuration
- π Project features
- ποΈ Import CSV file for places
- π Commands with scheduler
- π¬ Run the application tests
- π Logging channels
- π Aditional libraries used in project
- ποΈ Project TODOs
Laravel project using RapidAPis for scraping data from hosts:
- php: v7.3|v8.0
- db: mysql
- laravel/framework: v8.75
- bootstrap libary: v4.6.1
- jquery libary: v3.5.1
- datatables libary: v1.11.3
- When project is cloned first copy
.env.example
and create.env
file in root of project. - Update
composer.json
usingcomposer update
- Set the application key
php artisan key:generate
- Setup database configuration in
.env
file and migrate databasephp artisan migrate
- Create new user use seed
php artisan db:seed --class=UserTableSeeder
- Setup RapidAPI confiugration values in .env file
RAPIDAPI_HOTEL_HOST, RAPIDAPI_WEATHER_HOST, and RAPIDAPI_KEY
RAPIDAPI_HOTEL_HOST="x-rapidapi-host: hotels4.p.rapidapi.com" RAPIDAPI_WEATHER_HOST="x-rapidapi-host: community-open-weather-map.p.rapidapi.com" RAPIDAPI_KEY="x-rapidapi-key: keyNum"
- Setup Google maps static API
GOOGLE_MAPS_STATIC_API_KEY="API_KEY"
- Run/Serve the application on the PHP development server use command
php artisan serve
- Use csv file that is located
public/csv/placesdata.csv
and register or login with seed user, then in homepage go to Import CSV data for places link and use upload form
- Login/Register user
- Create/Import CSV for places(columns: country, city, and date)
- Export data form with inputs: country, city, date (Fill all inputs store as place and find suggested three hotels or check daily weather forecast using RapidAPI.)
- Hotel list with filter(search by country or city)
- Places table list
- View single place with list of hotels and weather information
- Commands for exporting data (scrape data for hotels and weather and store to database)
- Unit testing project requests(it covers all projects requests)
- Custom logs for export data from RapidAPI (
place_export.log, place_export_data.log and rapid_api.logs
)
In this page(/import) use form upload csv file that needs to have columns country, city and date. When csv is uploaded use import button to store all data into places table.
When project is setup and places are imported via CSV upload or created manualy there is option to automaticly scrape RapidAPI data from hotels and weather.
Use commands:
To see list of command type in command prompt php artisan
and find in list of commands:
php artisan export_manual:hotels
php artisan export_schedule:hotels
This command is used when you want to select data with conditions:
- Select places with single/all country or city
- Additional store data weather if place have hotels
This command is used in scheduler it export all data using RapidAPIs for hotels and weather
Hint: To see scheduler list command use php artisan schedule:list
To active scheduler use command php artisan schedule:run
Hint: Schaduler time interval 6 hours when againg command will be executed
Project have intergated unit tests for all requests in project
Tests are separated in files(/tests/Feature/*
):
- UserTest.php
- HotelTest.php
- ImportTest.php
- PlaceTest.php
Run tests using command php artisan test
or php artisan test --filter UserTest
Laravel provides robust logging services that allow you to log messages to files, the system error log.
In config/logging.php
there is registered new channels that runs when RapidAPI is used:
- placeexportlog (storage/logs/place_export.log)
- placeexportdatalog (storage/logs/place_export_data.log)
- rapidapilog (storage/logs/rapid_api.log)
- Composer packages:
- glhd/laravel-dumper v0.1.0
- Monarobase Country List v3.2
- Bootstrap CSS framework libary version 4.6
- Datatables
- Google maps API static images
- Masonry grid layout library
- β Implement new api endpoint using RapidAPI:
- β After exporting covid data from RapidAPI create migrations(new table covid_status_counties)
- β When migrations are done update places for additional data about country covid status
- β
Hotels list with filter - β
Unit test hotels requests