It is a set of scripts intended to crawl the currency exchange rates at the Central Bank of the United Arab Emirates. The collected rates are being written to a MongoDB database and can be retrieved via REST interface.
Open this page. There is a JavaScript application that allows a user to get rates for any date within the current month.
Besides it, there are Excel and PDF files with historical data for previous periods (take a look at the very bottom). Usually, the bank makes a new one when a month ends. For instance, if today is April, the bank published the Excel file for March for sure.
So, it is possible to get rates in two different ways:
- Currency rates for this month can be received via REST service, which lies behind the JavaScript application I mentioned above.
- Currency rates for previous periods can be retrieved as well from published files with historical data.
All the settings you can apply can be found in config.yaml file.
If you need any help, take a look at comment sections in the file: I was trying to make as many of them as possible.
Well, long story short:
- Setup periodical running for load_current.py
- Do the same for load_history.py
- Start the REST service using api.py.
All Python dependencies listed here.
More details are below.
Current rates are the rates for the short period of time back from this moment. For instance, two weeks.
The load_current.py script puts into the database currency rates, which are possible to crawl via the REST service of the bank. The bank used to publish actual rates approximately at 6:00 PM, so you can execute this script every evening at 8:00, for instance.
For instance, this is an example of a URL the script can crawl to get currency rates for February 17, 2023.
Historical rates are the rates that have been published on the bank website in Excel files.
This script tries to find the files on the respective page, then parses each one and puts the collected rates into a database.
You are supposed to start this script from time to time to be sure that if the bank changes something without warning, you will see the changes in your database. However, you can execute the script only once (for instance, if you just want to load all currency rates that are possible to get).
It is a simple Flask app you may run via gunicorn, uwsgi, or unit. It enables any application to get currency rates accumulated in the MongoDB database.