oZKS Tasks

[Ercan Ozturk]

Summary

#189 has removed the need for storing states-per-epoch and revised the HistoryTreeNode (referred to as node below) to contain the only (current) state (aka hash), significantly reducing the storage requirements. This issue is a tracker for the next steps (duplicate of this comment).

Important Changes

• Each node keeps a label of their left and right children, and its hash.
• For leaf nodes the hash is over the initial digest and the label. Initial hash is the hash of public key (and epoch, see next steps).
• For non-leaf nodes, the hash is over the left and right child, and the label.
• For the root, we use the empty node hash: over the empty value and empty label.
• The last_epoch has been kept to be used in append-only proofs whereas birth_epoch is not needed anymore.
• For each call, set_child and update_node_hash writes the updated nodes to the storage for consistency. See below for optimizations over this.
• Epochs in the functions have been kept for adding them to the hash later.
• Storage has been removed from function parameters if the function operated on state or children.

Next Steps:

• Clean up: We should remove the dead-code, debug output, unused HistoryChildState and HistoryNodeState, along with the storage layer code for these nodes.
• Remove birth epoch: Not needed anymore
• Clippy: Make sure Clippy passes.
• Merge with eozturk1:optimized-azks:
• Difference between get_leaf_node and get_leaf_node_without_hashing: How should they differ?
• Addition of epoch to the leaf: Leaves should include the latest epoch information in their initial hash. This will allow us to track when a leaf was inserted into the tree.
• Updating history/audit/lookup tests:
  • tests::test_directory_polling_azks_change
  • tests::test_limited_key_history
  • tests::test_simple_audit
  • tests::test_simple_key_history
  • tests::test_simple_lookup
  • akd_client tests::test_simple_lookup
  • akd_client tests::test_simple_lookup_for_small_tree
  • akd_client tests::test_tombstoned_key_history
• Updating proof tests:
  • append_only_zks::tests::test_append_only_proof
  • append_only_zks::tests::test_append_only_proof_tiny
  • append_only_zks::tests::test_append_only_proof_very_tiny
  • append_only_zks::tests::test_membership_proof_intermediate
  • append_only_zks::tests::test_membership_proof_permuted
  • append_only_zks::tests::test_nonmembership_proof
  • akd_client tests::test_history_proof_single_epoch
  • akd_client tests::test_history_proof_multiple_epochs
• High level:
  • memory_tests::test_directory_operations
  • mysql_tests::test_directory_operations
  • mysql_tests::test_lookups
  • mysql_db_tests::test_mysql_db
• Fix: #144
• #213
  • tests::test_read_during_publish
• Renaming HistoryTreeNode as TreeNode: No history, only current.

Improvements

• Tests: We should add more tests for small trees and nodes with non-uniform distribution to increase test coverage for tree creation.
• Minimizing storage writes: If a node is not written to the storage as soon as it is updated, a dirty-read may occur for consecutive operations. We might be able to minimize these writes by allowing short-term inconsistencies by restructuring the code. Another question to look into is how expensive are these instant-writes? (@afterdusk mentioned we use transactions to minimize the impact)
• Tree Visualization: Noticed it is hard to track the current state of the tree through logs. Maybe a visual representation could assist us in this?