vsbogd / hyperon-experimental

Experimental Hyperon versions

Overview

OpenCog Hyperon is a substantially revised, novel version of OpenCog - which is currently at an active pre-alpha stage of development and experimentation. One of the focuses in the Hyperon design is a successor to the OpenCog Classic Atomese language with clear semantics supporting meta-language features, different types of inference, etc. What we have landed on is an "Atomese 2" language called MeTTa (Meta Type Talk).

In order to get familiar with MeTTa one can visit MeTTa website and watch video with different MeTTa examples explained. The examples of MeTTa programs can be found in ./python/tests/scripts directory. Please look at the Python unit tests to understand how one can use MeTTa from Python. More complex usage scenarios are located at MeTTa examples repo. A lot of different materials can be found on OpenCog wiki server. Also see MeTTa specification.

If you want to contribute the project please see the contributing guide first. If you find troubles with the installation, see the Troubleshooting section below. For development related instructions see the development guide.

Using the latest release version

It is the most simple way of getting MeTTa interpreter especially if you are a Python developer. The following command installs the latest release version from PyPi package repository:

python3 -m pip install hyperon

Another way is using released Docker image:

docker run -ti trueagi/hyperon:latest

After installing package or starting Docker container run MeTTa Python based interpreter:

metta-py

Using Docker you can also run Rust REPL:

metta-repl

Using latest development version

Docker

A docker image can be used as a ready to run stable and predictable development environment. Docker 26.0.0 or greater version is required to build image manually.

Build Docker image from a local copy of the repo running:

docker build -t trueagi/hyperon .

Or build it without local copy of the repo running:

docker build \
    --build-arg BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 \
    -t trueagi/hyperon \
    http://github.com/trueagi-io/hyperon-experimental.git#main

Use --target build option to create an image which keeps the full build environment and can be used for developing interpreter:

docker build --target build -t trueagi/hyperon .

If the docker image doesn't work, please raise an issue.

Manual installation

Prerequisites

• Install the latest stable Rust, see Rust installation page. Make sure your PATH variable includes $HOME/.cargo/bin directory after installing Rust (see the Notes at the installation page).

  Requirements for building C and Python API

  • Python3 and Python3-dev (3.7 or later)
  • Pip (23.1.2 or later)
  • GCC (7.5 or later)
  • CMake (3.24 or later)

  To support Git based modules (enabled by default):

  • OpenSSL library
  • Zlib library

• Install cbindgen:

cargo install --force cbindgen

• Install Conan and make default Conan profile:

python3 -m pip install conan==2.5.0
conan profile detect --force

• Upgrade Pip to the required version:

python3 -m pip install pip==23.1.2

Build and run

Rust library and REPL

Build and test the Rust binaries:

cargo test

The experimental features can be enabled by editing Cargo.toml file before compilation or by using --features command line option. See comments in the [features] section of the file for the features descriptions.

Run examples:

cargo run --example sorted_list

Run Rust REPL:

cargo run --bin metta-repl

You can also find executable at ./target/debug/metta-repl.

To enable logging during running tests or examples export RUST_LOG environment variable:

RUST_LOG=hyperon=debug cargo test

Running benchmarks requires nightly toolchain so they can be run using:

cargo +nightly bench

Generate docs:

cargo doc --no-deps

Docs can be found at ./target/doc/hyperon/index.html.

C and Python API

Setup build:

mkdir -p build
cd build
cmake ..

To run release build use -DCMAKE_BUILD_TYPE=Release cmake flag.

Build and run tests:

make
make check

Running Python and MeTTa examples

In order to run examples you need to install the Python module. Please ensure you built C and Python API first. Then execute the following command in the top directory of repository:

python3 -m pip install -e ./python[dev]

After this one can run unit tests within python directory using pytest:

pytest ./tests

One can run MeTTa script from command line:

metta-py ./tests/scripts/<name>.metta

Run REPL:

cargo run --features python --bin metta-repl

You can also find executable at ./target/debug/metta-repl.

Running the REPL with Python support in a Python virtual environment like PyEnv or Conda requires additional configuration. See troubleshooting

Logger

You can enable logging by prefixing the MeTTa command line by

RUST_LOG=hyperon[::COMPONENT]*=LEVEL

where

• [::COMPONENT]* is a, possibly empty, sequence of modules and submodules of hyperon, such as ::metta, ::metta::runner, ::common, ::common::multitrie, etc.
• LEVEL is the log level. Possible log levels are: error, warn, info, debug and trace.

For example, to log all hyperon messages at the debug level and below, while running script.metta, you may type:

RUST_LOG=hyperon=debug metta script.metta

Or, to log all hyperon messages at the trace level and below, restricted to module metta and submodule types, you may type:

RUST_LOG=hyperon::metta::types=trace metta script.metta

By default all log messages are directed to stderr.

Troubleshooting

Conan claims it cannot find out the version of the C compiler

If you see the following cmake output:

ERROR: Not able to automatically detect '/usr/bin/cc' version
ERROR: Unable to find a working compiler

Try to create the default Conan profile manually:

conan profile detect --force

If it doesn't help, then try to manually add compiler, compiler.version and compiler.libcxx values in the default Conan profile (~/.conan2/profiles/default). For example:

compiler=gcc
compiler.version=7
compiler.libcxx=libstdc++

Rust compiler shows errors

Please ensure you are using the latest stable version:

rustup update stable

Importing hyperon Python module fails

If importing the hyperon module in Python

import hyperon

returns the error:

ModuleNotFoundError: No module named 'hyperonpy'

Please ensure you have installed the Python module, see Running Python and MeTTa examples.

Rust REPL cannot load Python library

The REPL needs a path to the libpython library in the current environment. This can be done one of two ways:

On Linux

Use patchelf on resulting REPL binary to link it with libpython.so

ldd target/debug/metta-repl | grep libpython ; to find <libpython-name>
patchelf --replace-needed <libpython-name> <path-to-libpython-in-virtual-env> target/debug/metta-repl

This must be redone each time the repl is rebuilt, e.g. with cargo build.

Set the LD_LIBRARY_PATH environment variable prior to launching metta-repl

export LD_LIBRARY_PATH=<path-to-libpython-directory-in-virtual-env>

On Mac OS

Use install_name_tool to change the REPL binary's link path for libpython

otool -L target/debug/metta-repl | grep libpython ; to find <libpython-name>
install_name_tool -change <libpython-name> <path-to-libpython-in-virtual-env> target/debug/metta-repl

This must be redone each time the repl is rebuilt, e.g. with cargo build.

Set the DYLD_FALLBACK_LIBRARY_PATH environment variable prior to launching metta-repl

export DYLD_FALLBACK_LIBRARY_PATH=<path-to-libpython-directory-in-virtual-env>

This can be done in your ~/.bashrc file if you don't want to do it each time you launch the REPL.

For more information about linking libpython, see #432.

Development

Structure of the codebase

Main library libhyperon.rlib is written in Rust language, it contains core API which can be used from other Rust projects. Source code of the library is located under ./lib directory. It is a plain Rust project which can be built and tested using Cargo tool.

In order to provide API for platforms and languages other than Rust there is a C API export library libhyperonc. Source code of the library is located under ./c directory. The library contains Rust C API bindings and depends on libhyperon.rlib library. Native library is compiled using Cargo, C headers are generated using cbindgen tool.

Source code of the Python integration library is located under ./python directory. It contains two main parts. First part is a native Python library libhyperonpy which is written using pybind11, it converts Python API calls into C API calls and vice versa. Second part is a Python library hyperon which uses libhyperonpy as a proxy for a C API calls.

All components which depend on libhyperonc are built using CMake build tool in order to manage dependencies automatically.

The diagram below demonstrates main components and dependencies between them: Source code of the diagram

Language support for IDEs

Different IDEs may require different tweaks to support the languages used in the codebase. The language servers which we use for development are:

• rust-analyzer;
• clangd, generate compile commands for the clangd using cmake variable:

  cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=Y ..
  

• Python LSP server.