fabiocmazzo/spago

If you like the project, please ★ star this repository to show your support! 🤩

A Machine Learning library written in pure Go designed to support relevant neural architectures in Natural Language Processing.

spaGO is self-contained, in that it uses its own lightweight computational graph framework for both training and inference, easy to understand from start to finish.

Features

Natural Language Processing

Memory-efficient Word Embeddings (with badger key–value store)
Character Language Models
Recurrent Sequence Labeler with CRF on top ( e.g. Named Entities Recognition)
Transformer models:
- Masked Language Model
- Next sentence prediction
- Tokens Classification
- Text Classification (e.g. Sentiment Analysis)
- Question Answering
- Natural Language Inference (i.e. Textual Entailment)
- Zero-Shot Text Classification
- Text Similarity
- Machine Translation

Internal Machine Learning Framework

Automatic differentiation:
- Define-by-Run (default, just like PyTorch does)
- Define-and-Run (similar to the static graph of TensorFlow)
Optimization methods:
- Gradient descent (Adam, RAdam, RMS-Prop, AdaGrad, SGD)
- Differential Evolution
Neural networks:
- Feed-forward models (Linear, Highway, Convolution, ...)
- Recurrent models (LSTM, GRU, BiLSTM...)
- Attention mechanisms (Self-Attention, Multi-Head Attention, ...)
- Recursive auto-encoders

Additional features

spaGO is compatible with pre-trained state-of-the-art neural models:

Hugging Face Transformers
Flair sequence labeler architecture

Usage

Requirements:

Clone this repo or get the library:

go get -u github.com/nlpodyssey/spago

spaGO supports two main use cases, which are explained more in detail in the following.

CLI mode

Several programs can be leveraged to tour the current NLP capabilities in spaGO. A list of the demos now follows.

The Docker image can be built like this.

docker build -t spago:main . -f Dockerfile

Library mode

You can access the core functionality of spaGO, i.e. optimizing mathematical expressions by back-propagating gradients through a computational graph, in your own code by using spaGO in library mode.

At a high level, it comprises four main modules:

Matrix
Graph
Model
Optimizer

To get started, look at the implementation of built-in neural models, such as the LSTM. Don't be afraid, it is straightforward Go code. The idea is that you could have written spaGO :)

You may find a Feature Source Tree useful for a quick overview of the library's package organization.

There is also a repo with handy examples, such as MNIST classification.

Current Status

We're not at a v1.0.0 yet, so spaGO is currently work-in-progress.

However, it has been running smoothly for a quite a few months now in a system that analyzes thousands of news items a day!

Besides, it's pretty easy to get your hands on through, so you might want to use it in your real applications.

Early adopters may make use of it for production use today as long as they understand and accept that spaGO is not fully tested and that APIs might change.

Known Limits

Sadly, at the moment, spaGO is not GPU friendly by design.

Contributing

We're glad you're thinking about contributing to spaGO! If you think something is missing or could be improved, please open issues and pull requests. If you'd like to help this project grow, we'd love to have you!

To start contributing, check the Contributing Guidelines.

Contact

We encourage you to write an issue. This would help the community grow.

If you really want to write to us privately, please email Matteo Grella with your questions or comments.

Acknowledgments

spaGO is a personal project that is part of the open-source NLP Odyssey initiative initiated by members of the EXOP team. I would therefore like to thank EXOP GmbH here, which is providing full support for development by promoting the project and giving it increasing importance.

fabiocmazzo / spago