Optimized azks implementation
Jasleen1 opened this issue · comments
The current implementation is based on a minimally optimized version of the construction of a History Tree data structure from the SEEMless paper (see figure 2 in this https://eprint.iacr.org/2018/607.pdf). This data structure stores every state ever of all tree nodes, leading to significant storage costs. We can optimize this away by making the following change: if the value stored at a given leaf in the tree was previously val
, we now replace it with hash(val, t)
where t
is epoch when that leaf was first inserted. Now, in order to verify that a leaf was inserted at the correct time, a client only needs to check that the correct timestamp was hashed in the leaf. The auditors must still audit all consecutive epochs for the append only property and ensure that the epoch hashed in any new leaf is correct, in order to maintain the same security guarantees as SEEMless and our current implementation.
Fortunately, most of the APIs remain the same with only minor changes. At a high level this requires the following code changes:
HistoryTreeNode
no longer needs to exist, we can just have aTreeNode
with main APIs
(a)insert_single_leaf(&mut self, leaf: Self)
: Inserts a leaf without a hash using a helper function.
(b)insert_single_leaf_with_hash(&mut self, leaf: Self)
: Inserts a leaf and updates the hash up the tree.
(c)insert_single_leaf_helper(&mut self, leaf: Self, hashing_flag: bool)
: Inserts a single leaf with or without hashing as needed.
(d)update_hash(&mut self)
: Updates the hash of this node and sets it at the parent, if relevant.Azks
: This should remain almost the same, except, theget_append_only_proof
will probably need to be run as part of a call tobatch_insert_leaves
, which will need to keep track of the unchanged nodes. This is the main algorithmic complexity in this entire change. Additionally the proofs themselves will contain the epoch when some node was added, so the value of the leaf nodes should somehow also store the epoch when the leaf was created.Directory
: The only change here will be in theaudit
which will now only allow auditing the latest update. One design decision here is whether we would like to store the audit proofs for a few of the most recent epochs.- The data structure
LookupProof
should now include the epoch when a leaf was inserted whenever a membership proof is sent. The corresponding changes also need to be made toAuditProof
andKeyHistoryProof
. - All the corresponding client and auditor verification functions need to be modified.
Note that a similar data structure has been implemented and open sourced by Microsoft Research, which might be helpful. See: https://github.com/Microsoft/oZKS.
Updates on the get_append_only_proof
changes mentioned above:
I had a discussion about this with @eozturk1. We found that if we keep the last_epoch
field in the TreeNode
, we can use it to compute parts of the tree which are unchanged since some epoch start_epoch
. However, the end_epoch
of the audit would need to be accounted for by including only leaves inserted upto end_epoch
Thus the function get_append_only_proof_helper
can remain largely unmodified.
This affects (2) and (3) above. The get_audit_proof
of the Directory
would also now only need minor changes to account for the new TreeNode
representation.