This project currently developed on Python restfull web service under Windows OS, to run this project you should have the requirments:
- Python 3.7.3
- Beautiful Soup: (error tolerant) HTML parsing software This software is currently developed using bs4(4.8.2)
- Other Python modules (requests, sys, Flask, jsonify)
Make sure you have Python, BeautifulSoup & Flask installed
pip install beautifulsoup4
pip install Flask
-
Clone the project from github repository
git clone https://github.com/shadhin-int/Wikipedia-Movie-Data-Gateway.git -
Browse into the repo root directory (cd Wikipedia-Movie-Data-Gateway)
-
Install the requirement(pip install -r requirements.txt)
-
Now, follow below instructions,
To fetched and parsed from sources, and inserted into the database, follow below commands
python application.py parse
Note : This command will drop all the database and scraped data again. Data already scraped so if you want to scraped data again just run this command otherwise don't need to run this command.
To run api server, follow below commands
python application.py serve
-
For List of Academy Award-winning films info (Film Name, Year, Awards and Nominations)
http://127.0.0.1:5000/GetMoviesInfo -
For list of Academy Award-winning films details info(parse 10 data from the right sidebar from individual link)
http://127.0.0.1:5000/GetMoviesDetailsInfo -
For list of Academy Award-winning films details info by movie id
http://127.0.0.1:5000/GetMoviesInfoById/1
- Scraped data from wikipedia from provied link - Done
- Stored data in SQLite database using Flask - Done
- A rest api client that will provide Movies info - Done
- A rest api client that will provide Movies deatils info - Done
- A rest api client that will provide Movies info by movies id - Done
- Generate data from movies.csv and ratings.csv file and store into existing database - Done
- Calculate average rating and rating givers for individual movie - Done