So practically want we want to do is a Task Manager.
- In which we are going to have 3 different types of tasks to request/send data:
- API:
- request information using a url
- send data to a post url
- DB:
- Requests information from a db
- Save information into a specific db
- Files:
- Get the file from a directory or an url and reading its content
- Create a file and save it into a directory or send it to a url
- The tasks defined above can be executed with different arguments, so there should be configuration objects that can be reused by those tasks, those configuration objects can contain different arguments like the URL, or db query, file dir/url or others.
- Every task execution information should be saved: which configuration was used, which tasks was run, time at which the task was run along with the result
- Tasks can be scheduled
- There are 2 types of scheduler entries, frequency(every n secs) and crontab(defining specific time)
- The scheduler entries information should be saved into a DB, but there is also the possibility of running a scheduler without having a db
- And more details about what I want as a user:
- As a user I can ask for:
- All the tasks I have
- Tasks by id
- All the tasks I have filter by type(api, db, files)
- All the tasks I have based on a time of period
- All the configuration I have
- All the schedulers entries I have
- As a user I can ask to:
- Schedule tasks every task schedule is scheduler entry
- Edit a schedule ?
- Run a task
- Save/Edit Configurations
- As a user I can ask to delete:
- Tasks by type
- Tasks by id
- Tasks run in a period of time
- Configurations that are not used by any tasks
- All Scheduler entries
- Scheduler entry by id
- As a user I can ask for:
# create virtualenv
virtualenv --python=python3
# activate the virtualenv
. venv/bin/activate
# install requirements
pip install -r requirements.txt
pip install -r requirements.dev.txt
Setup and run the application
export FLASK_APP=main.py
export FLASK_ENV=development
# run the flask development server
flask run
Setup if using pycharm
- Edit run configuration
- Add flask config and set the following
- script path: to venv/bin/flask
- Parameters: run
- Envirnoment variables:
FLASK_APP=main.py; FLASK_ENV=development; FLASK_DEBUG=1
- Working directory: root directory of task_scheduler app, for instance: /home/ubuntu/prog102-AT-projects/task-scheduler-at
The app main database uses mongo. See docs/Mongo-schema.md
# with virtualenvironment activated
pip3 install tox
# execute
tox
bash build-and-test.sh
Be sure to modify config files first, rename task-scheduler/configs/config.dev.sample.json
to task-scheduler/configs/config.dev.json
and modify the configs if needed, for instance:
{
"app_db": {
"name": "task_scheduler",
"username": "",
"password": "",
"port": 27017,
"host": "mongo"
},
"loglevel": "DEBUG",
"secret_key": "some secret dev!"
}
App and redis and mongo containers can be built by using docker-compose:
docker-compose up
Then at http://localhost:5000
app can be accessed
Additionally, for testing purposes Web flask app can also be built individually:
docker build -t task-scheduler-at-1 .
docker run -it --rm --name task-scheduler-at-1 task-scheduler
# then should run mongo and redis containers as well