masak1112 / workflow_parallel_frame_prediction

Workflow for Frame Prediction by Parallel Deep Learning

Workflow for parallel deep learning

This project implements a workflow for parallel deep learning to predict the 2m temperature based on Severin's master thesis code link thesis link .

The workflow consists of a sqeuence of steps (Data Extraction, Data Preprocessing, Training and Data Postprocess)to implement video prediction, and In each step try to Parallel for accelerating the whole prediction process.

The wokflow have been tested on the supercomputers from JSC, JURECA and JUWELS

Requirement:

• Keras
• Horovod
• Python3.6
• mpi4py

Usage

1. Clone or download this repository,
2. Install the required modules/packages on JURECA/JUWELS.

source packageInstallation.sh

3. Configure your input directory, output and log directory in .dat file for each step.

4. Run .sh file for submitting job

Workflow example