killingbear999 / ICLSTM

This work proposes an Input Convex LSTM neural network for real-time optimization.

Input Convex LSTM: A Convex Approach for Fast Model Predictive Control

Zihao Wang, Donghan Yu, Zhe Wu
Paper: https://arxiv.org/abs/2311.07202 (This version is outdated, we are working on the updated version)

Requires: Python 3.11.3, Tensorflow Keras 2.13.0, Pyipopt, Numpy, Sklearn
File description:

• docker.pptx includes the instruction on how to install Pyipopt into Docker on your laptop.
  
• ICLSTM_toy_examples.ipynb demonstrates the input convexity of ICLSTM in some 3D toy examples.
  
• Under CSTR subfolder:
  1. CSTR_ICLSTM.ipynb and CSTR_NNs.ipynb are used to train neural networks to learn the system dynamics.
     
• Under MPC subfolder:
  1. rnn.h5, lstm.h5, icrnn.h5, iclstm.h5 are trained RNN, LSTM, ICRNN, and ICLSTM respectively. You may regenerate the models using CSTR_ICLSTM.ipynb and CSTR_NNs.ipynb.
     
  2. mpc_rnn.ipynb, mpc_lstm.ipynb, mpc_icrnn.ipynb, mpc_iclstm.ipynb are used to integrate NNs into LMPC and solve the MPC optimization problem.

FYI:

• .ipynb files can be run on Jupyter Notebook or Google Colab.
• Pyipopt can be installed and run on Docker. mpc_rnn.ipynb, mpc_lstm.ipynb, mpc_icrnn.ipynb, mpc_iclstm.ipynb use Pyipopt.

The ICLSTM cell follows the structure as follows:

Specifically,
$f_t = ReLU[D_f(W_hh_{t-1}) + D_f(W_x[x_t,-x_t]) + b_f]$
$i_t = ReLU[D_i(W_hh_{t-1}) + D_i(W_x[x_t,-x_t]) + b_i]$
$c_{temp} = ReLU[D_c(W_hh_{t-1}) + D_c(W_x[x_t,-x_t]) + b_c]$
$o_t = ReLU[D_o(W_hh_{t-1}) + D_o(W_x[x_t,-x_t]) + b_o]$
$c_t = f_tc_{t-1} + i_tc_{temp}$
$h_t = o_tReLU(c_t)$
where

• $D_f$, $D_i$, $D_c$, $D_o$ are non-negative trainable scaling vectors
• $W_h$, $W_x$ are non-negative trainable weights (i.e., sharing weights across all gates)
• $b_f$, $b_i$, $b_c$, $b_o$ are trainable bias

Citation

If you find our work relevant to your research, please cite:

@article{wang2023input,
  title={Input Convex LSTM: A Convex Approach for Fast Lyapunov-Based Model Predictive Control},
  author={Wang, Zihao and Wu, Zhe},
  journal={arXiv preprint arXiv:2311.07202},
  year={2023}
}