How many transfer events fits in one parablock?
bredamatt opened this issue · comments
Bounds on POV size and relay block size (i.e. availability cores) are important factors to consider when measuring TPS, in particular for parachains. Indeed, the max data-rate of the relay chain is a function of how many availability cores exist, times the maximum POV size. Hence, given that a Transfer takes single_transfer_proof_size amount of bytes, then the total number of such Transfer events to fit on the relay chain at any given time can roughly be modelled as:
max_parachain_tps = ((max_POV_size/single_transfer_proof_size)*num_availability_cores)/parablock_inclusion_time
where parablock_time is the time it takes to include a parablock / candidate in the relay chain, which will vary based on whether sync-, or async-backing is used.
If we know how many Transfer events will fit in a particular POV, we can then start to think more clearly about how many sender binaries we should use to maximise throughput during the (s)TPS test.
This should definitely also relate to the weight associated with the Balances pallet transfers. I would like to see a way to go from weight to bytes, but don't think this is possible. So ideally, there is a way to calculate the size in bytes for each event given some Pallet. See: https://substrate.stackexchange.com/questions/518/what-does-pov-stand-for/519#519 for some more context.
After discussing with the FRAME team, it appears that the PoV bound is a bit more intricate than first expected, as there exists various PoV bounds at various points in time during the protocol execution:
- once pre-dispatch based on weight
- once with the proof-recorder
- once where the two above results get compacted and compressed for the final PoV size result
However, in this case the final one is what matters in the end so it makes sense to somehow test by setting weights in the Transfer call to 0 in the Balances pallet, then monitoring the PoV size that gets sent to relay whilst gradually increasing the number of Transfer events included in a candidate.
Thinking more about this, I think some work should be started on building out a separate tool, or macro that allows us to calculate the proof size for any dispatchable in substrate. This means we would have:
- weights to indicate execution time
- the tool to indicate proof size of a dispatchable
An alternative way to look at this would be in terms of execution time, which we currently can estimate using weights:
max_para_tps = ((max_exec_time/single_transfer_exec_time)*num_availability_cores)/parablock_inclusion_time
Naturally, there is some interplay between exec time and proof size, since even if you could theoretically fit a large number of transactions in terms of exec time, if few transfers consumes the entire PoV, then you'd still be limited to few transactions, and vice versa (i.e. if the transfer proof size is small, but the exec time large).
Thinking more about this, I think some work should be started on building out a separate tool, or macro that allows us to calculate the proof size for any dispatchable in substrate. This means we would have:
weights to indicate execution time
the tool to indicate proof size of a dispatchable
FRAME benchmarking does this. For example a TKA transaction has ~3.6KB proof size estimated here.
The exact size will depend on the size of the DB snapshot.
Thanks for guidance on this @ggwpez ! However, I am somewhat confused after seeing this: https://github.com/paritytech/smart-bench#running-benchmarks and their output:
0005: PoV Size=0130KiB(005%) Weight RefTime=0000088ms(017%) Weight ProofSize=3277KiB(064%) Witness=0119KiB Block=0011KiB NumExtrinsics=0048
...
where it is reported different numbers for the Weight ProofSize and the PoV size for a particular contract.
I suspect FRAME benchmarking gives weight Proof Size and not PoV Size?
Closing this.
Thanks for guidance on this @ggwpez ! However, I am somewhat confused after seeing this: https://github.com/paritytech/smart-bench#running-benchmarks and their output:
I dont know what smart-bench is doing or how it works. The Proof is additionally compressed when a block is being build, so I have no clue what exactly it is outputting or how it relates to FRAME. Maybe you can find out what it outputs.