Temporal Discrete Graph Updater

Description

The Temporal Discrete Graph Updater (TDGU) is a text-to-graph model that incrementally constructs temporal dynamic knowledge graphs from interactive text-based games.

Quickstart

Install dependencies.

# Install poetry https://python-poetry.org/
curl -sSL https://install.python-poetry.org | python3 -

# Clone the repository
git clone https://github.com/yukw777/temporal-discrete-graph-updater
cd temporal-discrete-graph-updater

# Install TDGU using poetry
poetry install

Download the dataset.

cd temporal-discrete-graph-updater
mkdir data
cd data
wget https://aka.ms/twkg/cmd_gen.0.2.zip
unzip cmd_gen.0.2.zip -d cmd_gen.0.2

Download the pretrained embeddings.

cd temporal-discrete-graph-updater
mkdir embedding
cd embedding
wget https://dl.fbaipublicfiles.com/fasttext/vectors-english/crawl-300d-2M.vec.zip
unzip crawl-300d-2M.vec.zip

Next, run the training script.

python scripts/main.py fit \
--model config/supervised/main-model.yaml \
--data config/supervised/data.yaml
--trainer.accelerator gpu

If you want to log to Weights & Biases, simply set the logger.

python scripts/main.py fit \
--model config/supervised/main-model.yaml \
--data config/supervised/data.yaml
--trainer.accelerator gpu \
--config config/supervised/wandb-logger.yaml

Configuration

This project uses PyTorch Lightning CLI as its configuration system. Take a look at config for the default configuration, which can be used to reproduce the main results of the paper.

Developing

Local Development Environment Setup

# Install poetry https://python-poetry.org/
curl -sSL https://install.python-poetry.org | python3 -

# Clone the repository
git clone https://github.com/yukw777/temporal-discrete-graph-updater
cd temporal-discrete-graph-updater

# Install TDGU with dev dependencies using poetry
poetry install --with dev

Testing

We use pytest to write and run tests. Once your local development environment is set up, simply run pytest to run tests. By default, slow tests are not run locally. If you want to run them locally, use pytest -m "".

All the tests are run as part of the CI/CD pipeline, so it's highly recommended that you make sure all the tests locally before opening a merge request.

Code Quality

We use various tools to ensure code quality. All of these tools can be run automatically at every commit by using pre-commit. Simply run pre-commit install after installing TDGU to set it up. You can also set these tools up with your favorite IDE. Please see below for instructions.

Code Linter

We use flake8 to lint our code. While you can manually run it, it's highly recommended that you set up your text editor or IDE to run it automatically after each save. See the list below for documentations on how to set flake8 up for various text editors and IDEs.

VSCode

Code Formatter

We use black to automatically format our code. While you can manually run it, it's highly recommended that you set up your text editor or IDE to run it automatically after each save. See the list below for documentations on how to set black up for various text editors and IDEs.

VSCode

Static Type Checker

We use mypy as our static type checker. It uses Python's type hints to perform static type checks. Please note that we use python3.9 which has some new type hints that do not exist in the previous versions. While you can manually run it, it's highly recommended that you set up your text editor or IDE to run it automatically after each save. See the list below for documentations on how to set mypy up for various text editors and IDEs.

VSCode

Git Branching Strategy

We follow a very simple git branching strategy where changes to the code base are made in a short-lived branch off the main branch. These branches should be merged as soon as possible via pull requests. A typical workflow is described below:

Create a new branch for your changes.

# No rules around branch names, but try to use a descriptive one.
git checkout -b <your-branch-name>

Make your changes while saving as frequently as necessary by making small commits.
While you should always try to write descriptive commit messages, at this step, it's not strictly necessary. So commit messages like wip or typo are OK here.
Clean up your commits using git interactive rebasing.
The goal of this step is to ensure all of the commits on your branch are self-contained with descriptive commit messages. You can do so by using git interactive rebasing. Here's a quick blog post and a short video on how to use it. If you'd like a more thorough documentation, you can check out this page. Here is a good blog post on how to write good commit messages.
Open a pull request (PR).
Now that your branch is all cleaned up, go ahead and open an PR. If your branch has only one self-contained commit, you don't have to do much since the title and description would be pre-filled. If your branch has multiple self-contained commits, make sure to summarize them in the MR similar to how you'd write a git commit message.
Fix the PR based on review comments.
Make sure to clean up your commits via git interactive rebasing. Your local branch may go out of sync with the remote branch at this step, and it's OK to force push git push origin HEAD --force to push the cleaned up branch with the fixes.
Merge the merge request.
Once the merge request is approved, go ahead and merge the merge request. Squashing is recommended (unless you know what you're doing).

yukw777 / temporal-discrete-graph-updater