Mastering Bitcoin

This repository contains example code related to the book Mastering Bitcoin 2nd Edition (book Github repo).

The code examples in the book are all in Python and C++, so I have ported them to Elixir. Since Elixir can't use third-party libraries outside of a mix project, each of the files referenced in the book (eg rpc_example.py) have been ported to an individual module (eg MasteringBitcoin.RPCExample) and their functions can be run inside an iex terminal.

I haven't been able to find Elixir libraries that wrap or provide a replacement for the following libraries:

So, for C++ and Python files, source code has been moved out into Elixir as much as possible, and only code that is responsible for calling language-specific Bitcoin libraries has been left in the source files.

API-like communication between Elixir and C++ is done using Cure, and Export is used for communication between Elixir and Python.

Blog Posts

More details about how I set all of this up and got Elixir communicating with Python and C++ can be found in the following blog posts I wrote:

Setup

git clone git@github.com:paulfioravanti/mastering_bitcoin.git
cd mastering_bitcoin
mix deps.get

The book says that:

If you're reading this book and interested in developing bitcoin software, you should be running your own node.

So, that means running a full Bitcoin node on your laptop, and needing to download the entire blockchain (> 100GB worth of transactions) on to your computer (there is no way around this). The book gives an example node configuration file for a resource-constrained system (Example 3-2), and that's what I used to not completely tie up my network and kill my bandwidth limits with Bitcoin traffic.

Some of the code examples in the book use very specific blockchain transactions which you may not have yet on your local Bitcoin node. Rather than wait however many days/weeks it would take to potentially get those specific transactions before beginning to code, my ported code samples have catch-all conditions that use other transactions from any available in the local blockchain to generate any desired outputs.

Installation

These instructions relate to what I had to do for Mac OS to get up and running (your mileage may vary with other operating systems):

Bitcoin

Use Homebrew to install Bitcoin packages needed for code examples in the book:

bitcoin: provides bitcoind (the Bitcoin daemon) and bitcoin-cli (the command-line interface that enables communication via Bitcoin's API)
libbitcoin: Various Bitcoin-related helper functions (libbitcoin-explorer will bring in this library)
libbitcoin-explorer: Libbitcoin's CLI tool. Provides the bx command
gcc: Needed to compile the C++ files in the priv/ directory
python: Needed to run Python files in the priv/ directory. Install either via brew or via a version control manager like asdf.

Install brew packages (includes the C++ libraries):

brew install bitcoin libbitcoin-explorer gcc

Since Pybitcointools is not maintained any more, in order to get things working, the library has to be installed using pip at the specific commit hash before the entire repository was deleted:

pip install git+https://github.com/vbuterin/pybitcointools.git@aeb0a2bbb8bbfe421432d776c649650eaeb882a5

The bitcoin package comes with a service that can start and stop the bitcoind daemon, so for exercises that require communication with a Bitcoin node, this will need to be running (but don't forget to turn it off when you don't need it, especially if you have metered internet!):

brew services start bitcoin
brew services stop bitcoin

Bitcoin Core

Bitcoin Core is software that contains an example implementation for a wallet, but more importantly, software for a Bitcoin full node that is needed to query against for the book exercises. It can be installed using Homebrew Cask:

brew cask install bitcoin-core

Once installed, use your favourite text editor to open up the configuration file located at
~/Library/Application\ Support/Bitcoin/bitcoin.conf to configure how much of your computer's resources the node should use (shown in Example 3-1 and Example 3-2 in the book).

NOTE: even if you use the resource-constrained config in Example 3-2, you must do the following:

add the txindex=1 line from Example 3-1 to it otherwise you won't be able to make queries on transaction (tx) IDs via the Bitcoin API (which is something the exercises in the book do).
remove the prune=5000 line since Prune Mode is incompatible with txindex. Attempting to use Prune Mode with txindex will result in an error.

Test local Bitcoin node

Once bitcoin and bitcoin-core are installed and brew services start bitcoin has been run, you should now be able to use bitcoin-cli to test that everything is working correctly:

bitcoin-cli getinfo

Bitcoin nodes run at http://localhost:8332 by default, and if you want you can also use curl (brew install curl) to send API requests to the node:

curl --user <user>:<password> --data-binary \
  '{"jsonrpc":"1.0","method":"getinfo","params":[]}' http://localhost:8332

Replace <user> and <password> with the relevant values from your
~/Library/Application\ Support/Bitcoin/bitcoin.conf file.

Example code config

Add your Bitcoin node's username and password to the config for the Elixir example code in the same way as the curl command above:

cp config/config.example.exs config/config.exs

Then, edit the bitcoin_url: "http://<user>:<password>@localhost:8332" line of the config.exs file and substitute out the user and password information.

Run the code

Start the Bitcoin service, open up an iex console, and run the functions. For example:

brew services start bitcoin
iex -S mix

Example of interacting with the Bitcoin node:

iex(1)> MasteringBitcoin.Client.getinfo()
{:ok,
 %{"balance" => 0.0, "blocks" => 375964, "connections" => 8,
   "difficulty" => 59335351233.86657, "errors" => "",
   "keypoololdest" => 1508115486, "keypoolsize" => 1000, "paytxfee" => 0.0,
   "protocolversion" => 70015, "proxy" => "", "relayfee" => 0.0001,
   "testnet" => false, "timeoffset" => -3, "version" => 140200,
   "walletversion" => 130000}}
iex(2)> MasteringBitcoin.RPCExample.run
375945
:ok

Example of running Elixir files that require compilation of C++ files (if there are any compilation failures on your system, you may need to tweak the cpp_compile command specified in config/config.exs):

iex(1)> MasteringBitcoin.Addr.run()
Public key: "0202a406624211f2abbdc68da3df929f938c3399dd79fac1b51b0e4ad1d26a47aa"
Address: "1PRTTaJesdNovgne6Ehcdu1fpEdX7913CK"
:ok

Example of running Elixir files that require Python libraries:

iex(1)> MasteringBitcoin.ECMath.run()
Secret: 7649...4213
EC point: {5723...1267, 5216...5415}
BTC public key: "037e...bc33"
:ok

Tests

Each of the exercises has a sanity test check to just make sure it works, and continues to work as expected.

Run the test suite

mix test

The test suite deliberately excludes tests for files that require a Bitcoin node to be running, as those tests require brew services start bitcoin to be run, and warming up the node can take a variable length of time, leading to potential test suite failures. However, they do actually work, and can be specifically included when running the test suite:

mix test --include bitcoin_server

mix test.watch is included in this project, and hence all the tests (excluding bitcoin_server tests), as well as Credo and Dogma checks, can be continuously run when file changes are made:

mix test.watch

Other Helpful Resources

Controlling a Bitcoin Node in Elixir

paulfioravanti / mastering_bitcoin