greyireland / pyth-agent

Publish data to the Pyth Network

Pyth Agent

Publish data to the Pyth Network.

Overview

This software runs a JRPC API server, which data providers should use to publish data. Publishing using this intermediate API server provides greater reliability, usability and security than sending transactions directly to an RPC node.

Note that only permissioned publishers can publish data to the network. Please read the publisher guidelines before getting started.

Build

Prerequisites: Rust 1.68 or higher. A Unix system is recommended.

# Install OpenSSL (Debian-based systems)
$ apt install libssl-dev

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$ rustup default 1.68 # Optional

# Build the project. This will produce a binary at target/release/agent
$ cargo build --release

Configure

The agent takes a single --config CLI option, pointing at config/config.toml by default. An example configuration is provided there, containing a minimal set of mandatory options and documentation comments for optional settings. The config file must exist.

The logging level can be configured at runtime through the RUST_LOG environment variable using the standard error|warn|info|debug|trace levels.

Key Store

If you already have a key store set up, you can skip this step. If you haven't, you will need to create one before publishing data. A key store contains the cryptographic keys needed to publish data. Once you have a key store set up, please ensure that the configuration file mentioned above contains the correct path to your key store.

# Install the Solana Tool Suite, needed for creating the key used to sign your transactions.
sh -c "$(curl -sSfL https://release.solana.com/v1.14.13/install)"

# Create the key store directory. This can be any location that is convenient for you.
PYTH_KEY_STORE=$HOME/.pythd

# Create your keypair (pair of private/public keys) that will be used to sign your transactions.
# Pyth Network will need to permission this key, so reach out to us once you have created it.
solana-keygen new --no-bip39-passphrase --outfile $PYTH_KEY_STORE/publish_key_pair.json

# Initialize the key store with the public keys of the Pyth Oracle Program on the network you wish to publish to.
PYTH_KEY_ENV=devnet # Can be devnet, testnet or mainnet
./scripts/init_key_store.sh $PYTH_KEY_ENV $PYTH_KEY_STORE

Run

cargo run --release -- --config <your_config.toml> will build and run the agent in a single step.

Publishing API

A running agent will expose a WebSocket serving the JRPC publishing API documented here. See config/config.toml for related settings.

Development

Unit Testing

A collection of Rust unit tests is provided, ran with cargo test.

Integration Testing

In integration-tests, we provide end-to-end tests for the Pyth agent binary against a running solana-test-validator with Pyth oracle deployed to it. Optionally, accumulator message buffer program can be deployed and used to validate accumulator CPI correctness end-to-end (see configuration options below). Prebuilt binaries are provided manually in integration-tests/program-binaries - see below for more context.

Running Integration Tests

The tests are implemented as a Python package containing a pytest test suite, managed with Poetry under Python >3.10. Use following commands to install and run them:

cd integration-tests/
poetry install
poetry run pytest -s --log-cli-level=debug

Optional Integration Test Configuration

• USE_ACCUMULATOR, off by default - when this env is set, the test framework also deploys the accumulator program (message_buffer.so), initializes it and configures the agent to make accumulator-enabled calls into the oracle
• SOLANA_TEST_VALIDATOR, systemwide solana-test-validator by default - when this env is set, the specified binary is used as the test validator. This is especially useful with USE_ACCUMULATOR, enabling life-like accumulator output from the pythnet validator.

Testing Setup Overview

For each test's setup in integration-tests/tests/test_integration.py, we:

• Start solana-test-validator with prebuilt Solana programs deployed
• Generate and fund test Solana keypairs
• Initialize the oracle program - allocate test price feeds, assign publishing permissions. This is done using the dedicated program-admin Python package.
• (Optionally) Initialize accumulator message buffer program initialize test authority, preallocate message buffers, assign allowed program permissions to the oracle - this is done using a generated client package in integration-tests/message_buffer_client_codegen, created using AnchorPy.
• Build and run the agent

This is followed by a specific test scenario, e.g. test_update_price_simple - a couple publishing attempts with assertions of expected on-chain state.

Prebuilt Artifact Safety

In integration-tests/program-binaries we store oracle and accumulator *.sos as well as accumulator program's Anchor IDL JSON file. These artifacts are guarded against unexpected updates with a commit hook verifying md5sum --check canary.md5sum. Changes to the integration-tests/message_buffer_client_codegen package are much harder to miss in review and tracked manually.

Updating Artifacts

While you are free to experiment with the contents of program-binaries, commits for new or changed artifacts must include updated checksums in canary.md5sum. This can be done by running md5sum in repository root:

$ md5sum integration-tests/program-binaries/*.json > canary.md5sum
$ md5sum integration-tests/program-binaries/*.so >> canary.md5sum # NOTE: Mind the ">>" for appending

Updating message_buffer_client_codegen

After obtaining an updated message_buffer.so and message_buffer_idl.json, run:

$ cd integration-tests/
$ poetry install # If you haven't run this already
$ poetry run anchorpy client-gen --pdas program-binaries/message_buffer_idl.json message_buffer_client_codegen