LeanInfer: Neural Network Inference in Lean 4

LeanInfer provides tactic suggestions by running LLMs through Lean's foreign function interface (FFI).

It is in an early stage of development. In the long term, we aim to integrate Lean and machine learning by providing a general and efficient way to run the inference of neural networks in Lean. The network can be of arbitrary model architectures and trained using arbitrary deep learning frameworks. After training, it is converted into the ONNX format, which can be run as a shared library using ONNX Runtime and integrated into Lean through FFI.

Requirements

Supported platforms: Linux and macOS (⚠️ maybe also Windows, but untested)
Clang (⚠️ GCC not supported)
Lean 4
ONNX Runtime for optimized inference in C++

Adding LeanInfer as a Dependency to Your Project

Edit lakefile.lean to add the dependency require LeanInfer from git "https://github.com/lean-dojo/LeanInfer.git" and package configuration option moreLinkArgs := #["-lonnxruntime", "-lstdc++"] (see this example). Run lake update for the changes to take effect.
Download the model (LeanDojo's tactic generator in ONNX format) into the root of the repo. If you have Git LFS, this can be done by git lfs install && git clone https://huggingface.co/kaiyuy/onnx-leandojo-lean4-tacgen-byt5-small. Otherwise, see here.
Add the ONNX Runtime source directory (the directory that contains onnxruntime_cxx_api.h) to the environment variable CPATH. Add the ONNX Runtime library directory (the directory that contains libonnxruntime.so or libonnxruntime.dylib) to LD_LIBRARY_PATH (Linux), DYLD_LIBRARY_PATH (macOS), and LIBRARY_PATH (all platforms). If you are using Lean in VSCode, also add these environment variables to the Lean4: Server Env setting in VSCode.
If your default C++ compiler is not Clang (e.g., in most Linux systems), add LLVM's libc++ directory (the directory that contains libc++.so) to LD_LIBRARY_PATH (Linux), DYLD_LIBRARY_PATH (macOS), and LIBRARY_PATH. If you are using Lean in VSCode, also add it to Lean4: Server Env.
Run lake script run LeanInfer/check and fix problems (if any). Finally, run lake build.

If you have problems building the project, our Dockerfile may be helpful as a reference. Note that it doesn't need Step 4 because the base Ubuntu image has no compiler pre-installed.

Using LeanInfer's Tactic Generator

After import LeanInfer, you can use the tactic suggest_tactics (see the image above and this example).

Questions and Bugs

For general questions and discussions, please use GitHub Discussions.
To report a potential bug, please open an issue. In the issue, please include your OS information and the exact steps to reproduce the error. The more details you provide, the better we will be able to help you.

Acknowledgements

llmstep is another tool providing tactic suggestions using LLMs. We use their frontend for displaying tactics but a different mechanism for running the model.

Citation

@misc{leaninfer,
  author = {Song, Peiyang and Yang, Kaiyu and Anandkumar, Anima},
  title = {LeanInfer: Neural Network Inference in Lean 4},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/lean-dojo/LeanInfer}},
}

Code Formatting

The C++ code in this project is formatted using ClangFormat. To format the code, run

clang-format --style Google -i ffi.cpp

tydeu / LeanInfer