MASTER WILL NOT WORK PAST PERIOD 321 DUE TO BUG IN WEB3.JS (issue), THIS ISSUE IS BEING ADDRESSED IN ALPHA TESTING OF OPTIMIZATIONS BRANCH AND WILL BE MERGED IN AFTER TESTING

EOS Snapshot Generator

This tool was created to aggregate the EOS ERC20 token distribution in it's entirety, acknowledge various scenarios, run validation on data and provide files representing balances in various states.

This tool can be used to generate snapshots for any period in a deterministic fashion until the EOS ERC20 token has been frozen. Once frozen, the tool will only produce deterministic final snapshots that represent the final state of the EOS ERC20 token distribution.

Installation
Glossary

Wiki

Utilities

Offline Key Generator and Validator

External links

Telegram Channel

Installation

Prerequisites

MySQL (local or remote)
Parity 1.7.8+
Node 8+

System requirements

8GB Ram Recommended, can make due with 4gb
SSD recommended, NVME to win the race. HDD read/write speeds are intolerably slow with parity. It can increase your parity sync time by 3-4x, and will increase the amount of time for snapshot to process by 2-3x.
Around 110GB of storage space, required for full parity chain.

1. Create Config File

In the root directory of this project you'll find a config.default.js file.
Copy config.default.js and rename the copy to config.js. The parameters are described in the file, but for your convenience they are also listed below in greater detail. DO NOT REMOVE OR RENAME config.default.js

2. MySQL

MySQL stores all the intricate details of the distribution and the contract. These instructions assume you already have MySQL configured locally or have a remote MySQL instance.

Add your mysql details to your newly copied config file.
Import schema.sql located in the ./bin directory of this project.

3. Start and Sync Parity

Start parity, it's imperitive that you start parity with --no-warp. If you have a pre-existing parity configuration file you've modified, you'll want to create a new configuration file for the snapshot.

Sample Config:

parity --mode active --tracing off --pruning fast --db-compaction ssd --jsonrpc-apis all --chain mainnet --no-warp --cache-size 2048

Important: Since Parity v1.7.8 --warp is enabled by default. If you fail to configure with no-warp you will have issues.

Notes

If you must use an HDD, be sure to change the --db-compaction parameter for parity to hdd, like so: --db-compaction hdd
You can adjust --cache-size as needed, this could provide some sync-speed improvements.

3. Configure The Snapshot Parameters

If you're taking a "final" snapshot, much of the configuration is automatic.

A final snapshot will be taken automatically if the crowdsale has ended and the tokens are frozen, and will override configured period. Block ranges for the snapshot will be determined by the opening block (first transaction from crowdsale contract) and the deterministic freeze block. This is to prevent user error.

If you're taking an "ongoing" snapshot, change "period" to the period for which you would like to generate a snapshot. Note If you put in a period that hasn't yet completed, the "last closed period" will override your choice.

Explore the options as defined here

4. Run the Snapshot

From the root directory of this project directory, run the following:

npm install (first time)
node snapshot.js --load_config

5. Connections

If MySQL and Parity are running, and Parity is synced, the script will start running.
If either MySQL or Parity are not connecting, it will keep trying every 5 seconds until a connection is established. Double-check your config and that both MySQL and Parity are running.
If Parity is not done syncing, it will tell you so, and let you know how far along it is. Once it's synced, it will continue.
If you encounter an error, see "troubleshooting"

6. Output

6a. Directory Structure

When complete, the script will output several files into the ./data directory.
Inside the data directory, will be a directory named after the period number you generated, for example: ./data/123. If it's a final snapshot, into the ./data/final directory.
Inside that directory, is directory named after a numerical index. This is so that if you run it multiple times, you don't overwrite another snapshot.
This functionality helps with verification of determinism and for development purposes.

6b. Files

snapshot.csv - This file contains the Ethereum Address, EOS Key and EOS Balance of every address that registered correctly with the contract up to the configured period, and has a balance greater than the value set by snapshot_minimum_balance (default:1)
snapshot-unregistered.csv - This file contains the Ethereum Address and EOS Balance of every address that either failed to register or registered incorrectly with the crowdsale contract up to the configured period.
distribution.csv - This file includes the Ethereum Address and EOS balance of the entire distribution up to the configured period, with no rules or validation imposed.
snapshot.json - This file contains information about the snapshot-session that created the above files. This file should not be used for verification, but can be used for debugging and to help identify indeterminism.

6c. Snapshot in Root Directory

If overwrite_snapshot is set to true, all the above files will be put into the root directory of the project. If you are doing any development on this project, it's suggested that you change this to false (it's enabled by default in the config.default.js file.)

Glossary

wallet balance - The wallet balance is refers to an address's EOS ERC20 token balance
unclaimed balance - An unclaimed balance refers to tokens that were never claimed by an address. Despite being unclaimed, these still belong to the contributing address.
reclaimed balance - A reclaimed balance refers to EOS ERC20 tokens accidentally sent to the EOSCrowdsale or EOSToken contract. The balances of these contracts are not included in distribution calculations, so it's imperitive these balances are calculated to have accurate supply values.
total balance - The sum of wallet + unclaimed + reclaimed balances. The total balance is what is included as a user's balance in snapshots.
registered address - A registered address is an address with a balance greater than or equal to the minimum_snapshot_balance that has correctly registered their ethereum address with the EOSCrowdsale contract using a valid EOS Public Key.
unregistered address - An unregistered address is an address with a balance greater than or equal to the - minimum_snapshot_balance that has either incorrectly registered or failed to register their address with an EOS Public Key.
freeze block - The freeze block is a deterministic value represented by the block number representing the period that tokens were frozen. This block will mark the last block for which actions sent to the crowdsale contract will be honored (such as registrations)
snapshot - A file containing EOS public keys and balances that can be imported during an EOSIO boot sequence.
snapshot-unregistered - A file containing Ethereum addressees and balances. This file could potentially be imported during an EOSIO boot sequence into the table of a contract that enables Ethereum based claiming.
liquid supply - The liquid supply represents total aggregate EOS ERC20 tokens that are presently in circulation and detected by snapshot script, after the crowdsale ends, the liquid supply should equal the total supply.
expected supply - Expected supply is a mathematically determined value representing what the script expects the liquid supply to equal. Liquid Supply should be within 0.00000001% of expected supply as a result of dust acquired by precision reduction.

IOVBlockChain / genesis