[Persistence] Export Trees Tool

[Olshansk]

Objective

Build a tool that exports the data in the Merkle Trees into a JSON file.

Origin Document

While working on #284, @Olshansk started building a tool that dumps all the data in the KV into a json file for easier debugging. Since the scope of that implementation grew very large, it is being moved to a separate task. The git diff of this code can be used as a starting point for further implementation: export_trees.txt

Goals

• Get visibility into the data in the Merkle Trees
• Understand how the data in the Merkle Trees is structured
• A simple tool that can be used by anyone
• Enable rapid debugging & understanding

Deliverable

• A debug command that makes the LocalNet nodes dump the Merkle Trees into a JSON file
• A make command that pulls
• A list of follow-up tasks to process & understand this data. For example:
  • Visualising the tree similar to https://github.com/celestiaorg/smt/pulls?q=is%3Apr+is%3Aopen+sort%3Aupdated-desc
  • Enabling other tools to process this data

Non-goals / Non-deliverables

• Adding new functionality to how the Merkle tree is computed

General issue deliverables

• Update the appropriate CHANGELOG
• Update any relevant READMEs (local and/or global)
• Update any relevant global documentation & references

Testing Methodology

• All tests: make test_all
• LocalNet: verify a LocalNet is still functioning correctly by following the instructions at docs/development/README.md

---

Creator: @Olshansk

[Olshansk]

Superseded by all the work related to persistence snapshoting by @deblasis and @dylanlott