Web tools
olalonde opened this issue · comments
@gmaxwell @zw @FredericHeem I've just released web tools to generate / validate trees. I'm hoping it could be useful for testing other implementation and possibly catching obvious bugs.
I'm still a step behind on this (need to add s11n to the Clojure version still) but the tool looks good --- maybe @bifubao could also have a look at this, since I haven't got around to eyeballing his code yet :) I'll have more to say once I've run through the process myself and can think concretely about getting the most out of the tool for standardisation/testing.
The #verify side looks great. Nice work.
To make the #generate side most useful --- sharing test input and letting exchanges/wallets swap implementations --- I think we'll want (1) to make the accounts list format a standard and a part of the interface to our implementations; (2) to implement one deterministic tree layout for testing interoperability purposes only. I think yours and mine both naively generate full trees but in slightly different ways (I don't use a dummy, you do) so perhaps we can sort the difference and use that.
To make the accounts list uniform we should probably quote-wrap the balances (I hear JSON and floats go together badly; #verify might want to change that too). We'll need to include the nonce, picking a suitable nonce size and encoding. 128 bits in hex? Perhaps @gmaxwell could comment.
So perhaps:
[
{
"user": "olalonde",
"balance": "4045.1234",
"nonce": "27ae41e4649b934ca495991b7852b855"
},
...
]Sounds good to me! The most important would be to get the root / partial tree formats to be standard but I agree a standard format for the other stuff would be very helpful as well for testing implementations.
For now, I'll quote wrap the balance/nonce and I can probably get rid of the "dummy" account.
Balance in BTC or satoshi ? Let's opt for satoshi to avoid rounding errors ?
In terms of the likelihood of bugs in implementations, I suspect it doesn't actually matter much because they should be ironed out pretty quickly if implementations fail to interact and won't verify --- especially if we're specific about what representation gets hashed in a conforming implementation.
My gut feeling is to prefer code that would be strict enough now but require the simplest, least error-prone changes if sub-satoshi-division happened later, and I'd guess that's x.y with exactly 8dp --- but it's not something I can get too religious about.
@zw why do you want to include nonce in accounts.json?
To me it would be more convenient if accounts.json only contains data directly pulled from production database ,and leave nonce generation to tree creation tool. It would also be easier to standardize nonce generation by doing this. Thoughts?
why do you want to include nonce in accounts.json?
Only to make tests deterministic. So, one way to do it is just to have the tree creation code add standard nonces if they're not there (for production), but use existing ones if they're present and produce a deterministic tree layout (for testing).
👍
You can now specify the "nonce" values in the accounts JSON file to get a deterministic tree. Note that this will not work if the number of account is odd because my implementation adds a dummy account with balance 0 / random nonce to get a complete tree. Planning get rid of this automatically added dummy account in a future commit.
My code should no longer add a dummy account. Trees are now perfectly deterministic if you specify a nonce for each user in the accounts JSON. For example:
[
{
"user": "olalonde",
"balance": 4045,
"nonce": "somenonce"
},
{
"user": "gmaxwell",
"balance": 6905,
"nonce": "somenonce"
},
{
"user": "satoshi",
"balance": 10000,
"nonce": "somenonce"
},
{
"user": "luke-jr",
"balance": 300,
"nonce": "somenonce"
},
{
"user": "sipa",
"balance": 3027,
"nonce": "somenonce"
},
{
"user": "codeshark",
"balance": 200,
"nonce": "somenonce"
},
{
"user": "gribble",
"balance": 15,
"nonce": "somenonce"
},
{
"user": "alice",
"balance": 0,
"nonce": "somenonce"
},
{
"user": "bob",
"balance": 12,
"nonce": "somenonce"
},
{
"user": "charlie",
"balance": 9427,
"nonce": "somenonce"
},
{
"user": "mark",
"balance": 462,
"nonce": "somenonce"
},
{
"user": "anax",
"balance": 12,
"nonce": "somenonce"
},
{
"user": "gavin",
"balance": 3020,
"nonce": "somenonce"
},
{
"user": "stacy",
"balance": 68,
"nonce": "somenonce"
},
{
"user": "justin",
"balance": 3,
"nonce": "somenonce"
},
{
"user": "einstein",
"balance": 39,
"nonce": "somenonce"
},
{
"user": "picasso",
"balance": 83,
"nonce": "somenonce"
}
]
@zw got rid of the dummy account and other than that, I do naively generate a complete tree for now. This can be left up to implementations but for testing, you are welcome to try generating trees as I do (warning: my tree generation code is not very idiomatic https://github.com/olalonde/blind-liability-proof/blob/master/lib/blproof.js#L59) :)