feat(durable): expose mode-generic foldable impl for Registry

[emturner]

• Closes TZX-142
• Part of TZX-113

What

Expose mode-generic Foldable for Registry, with Prove mode support for HashFold and MerkleProofFold for Database.

Why

Required for Foldable<HashFold> and Foldable<MerkleProofFold> for Prove mode registry - needed to integrate Prove mode into octez.

Fix a bug in test_verify_replays_prove_reads that was taking the hash from the normal mode, instead of the intended prove mode, registry.

The wiring up of a Proof type, with FromProof for registry, will follow.

Manually Testing

make all

Tasks for the Author

• Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes).
• Eliminate dead code and other spurious artefacts introduced in your changes.
• Document new public functions, methods and types.
• Make sure the documentation for updated functions, methods, and types is correct.
• Add tests for bugs that have been fixed.
• Explain changes to regression test captures when applicable.
• Write commit messages in agreement with our guidelines.
• Self-review your changes to ensure they are high-quality.
• Complete all of the above before assigning this MR to reviewers.

[github-actions]

Benchmark results for revision ae7f174:

Metric	Duration	TPS
Mean	1.513592684s	26.428
Worst	1.528820608s	26.164
Best	1.496599046s	26.727
Standard Deviation	±9.186542ms	±0.161

Full results

Run	Transfers	Duration	TPS
1	40	1.51416235s	26.417
2	40	1.528820608s	26.164
3	40	1.510428198s	26.483
4	40	1.523882938s	26.249
5	40	1.519040274s	26.332
6	40	1.497784825s	26.706
7	40	1.512545438s	26.445
8	40	1.51883873s	26.336
9	40	1.516753165s	26.372
10	40	1.525899999s	26.214
11	40	1.526802523s	26.199
12	40	1.513544131s	26.428
13	40	1.522391364s	26.274
14	40	1.496599046s	26.727
15	40	1.514810245s	26.406
16	40	1.503568895s	26.603
17	40	1.509893696s	26.492
18	40	1.504050418s	26.595
19	40	1.508012748s	26.525
20	40	1.50402409s	26.595

Compare the results above with those for the default branch.

[codecov]

Codecov Report

❌ Patch coverage is 53.84615% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.08%. Comparing base (14fe641) to head (9906132).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
durable-storage/src/database.rs	66.66%	3 Missing ⚠️
durable-storage/src/database/traced_database.rs	0.00%	3 Missing ⚠️

❌ Your patch check has failed because the patch coverage (53.84%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1069      +/-   ##
==========================================
- Coverage   90.17%   90.08%   -0.10%     
==========================================
  Files         130      130              
  Lines       28110    28199      +89     
  Branches    28110    28199      +89     
==========================================
+ Hits        25348    25402      +54     
- Misses       2013     2047      +34     
- Partials      749      750       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[victor-dumitrescu]

This is something that came up in an AI review, but there's an issue with theRegistry mode-independent hash fold implementation, which is that in Prove mode it will record the access to the registry size, which then changes the proof:

    kv_test!(test_hashing_prove_registry_does_not_record_reads, KV: BackgroundKeyValueStore, {
        let (_keepalive, repo) = KV::setup_repo();

        let key_a = Key::new(&[1]).expect("Size less than KEY_MAX_SIZE");
        let key_b = Key::new(&[2]).expect("Size less than KEY_MAX_SIZE");

        let mut registry = setup_size_2_registry::<KV>(repo);

        registry
            .database_mut(0)
            .expect("database at index 0 exists")
            .set(key_a, Bytes::copy_from_slice(b"foo"))
            .expect("Setting in Normal mode should succeed");

        registry
            .database_mut(1)
            .expect("database at index 1 exists")
            .set(key_b, Bytes::copy_from_slice(b"bar"))
            .expect("Setting in Normal mode should succeed");

        let prove_registry = registry
            .try_start_proof()
            .expect("Converting to prove mode should succeed");

        let proof_before_hash =
            octez_riscv_data::merkle_proof::proof_tree::MerkleProof::from_foldable(
                &prove_registry.databases,
            );
        let proof_before_hash_bytes =
            serialise(&proof_before_hash).expect("Serialising proof should succeed");

        let normal_root = Hash::from_foldable(&registry);
        let prove_root = Hash::from_foldable(&prove_registry);
        assert_eq!(prove_root, normal_root);

        let proof_after_hash =
            octez_riscv_data::merkle_proof::proof_tree::MerkleProof::from_foldable(
                &prove_registry.databases,
            );
        let proof_after_hash_bytes =
            serialise(&proof_after_hash).expect("Serialising proof should succeed");

        assert_eq!(
            proof_after_hash_bytes, proof_before_hash_bytes,
            "Proof should be unchanged by hashing"
        );
    });

[emturner]

@victor-dumitrescu - thank you so much for the spot! That's a really good point.

I've traced the issue ultimately to the fold as you mentioned - for now, I think the easiest thing is to have the fold be over the vector fold itself; which indeed does the right thing (of not recording the size access in prove mode). Especially given that (quite rightly) the unrecorded_* methods are private to the vector module.

This does mean that the arity for the registry goes from 2 -> 4, but I think that's okay; especially given we have the future follow up to move arity to 2 everywhere.

I've included your test, too - which now passes