Skip to content

Conversation

@freshair18
Copy link
Collaborator

@freshair18 freshair18 commented Dec 3, 2025

Significant resources on the node are currently devoted to maintaining "higher order" children<->father relations between blocks, within an array of relation stores.
Notably though, the only true usage of these stores beyond the base level, regards to building a pruning proof - where forward traversal on the DAG is desired and supplied by these stores. The pruning proof itself however ultimately only contains circa 2000 blocks per level. This PR hence computes these higher level stores on the fly during pruning proof construction, for only a limited set of relevant blocks, and discards these stores as a whole following this. This parallels a past PR #494 on ghostdag stores.

WIP:
(a) Currently old stores are still present in the node, just not updated, deleted, or used anywhere. A clean up remains to be done removing them from the codebase, as does a concentrated deletion of them from the datadir in any node updating to a new version.
(b) the transient relations store is currently constructed very naively, with multiple attempts per level until a sufficiently deep root for that level is found. Unlike the case of ghostdag store, much of the data of stores is shared between iterations. A smarter implementation would build upon the store created in the last iteration. Considering this only effects the (relatively) short and uncommon process of creating a pruning proof, this could wait for a subsequent PR.

Local Performance Initial Impressions (relatively weak node, windows):

  • Pruning of higher relations was previously isolated as a major local bottleneck in the pruning process
  • Accordingly, an x10-x11 improvement was observed in pruning speed following this PR, simply by not having to prune higher relations. Numerically, from pruning 1000 blocks every 70 seconds (i.e. not keeping up), to pruning 1000 blocks every 6-7 seconds
  • datadir size is about 13GB, which I believe is a decrease.
  • headers IBD appears to be about x3-x5 faster.
  • Pruning proof verification section of IBD feels a tad slower, no exact measurements currently. It remains a very short stage regardless.

}
}
// Populate ghostdag_store and relation store (on a per level basis) for every block in the proof
for (level, headers) in expanded_proof.iter().enumerate() {
Copy link
Collaborator

@coderofstuff coderofstuff Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you only need to do this now for level = 0

so no outer for loop on all levels anymore and instead just look at level 0 directly to fill the relations there.

@freshair18
Copy link
Collaborator Author

freshair18 commented Dec 5, 2025

Finished (a).

(b) is probably best differed for a subsequent PR.

Wondering about keep_relations during pruning - I believe the logic as a whole remains necessary- but I'm not sure if the name keep_relations is really descriptive for what it does anymore.

@freshair18
Copy link
Collaborator Author

Note: there appears to currently at times be a deadlock of sorts, which results in Block relay ceasing activity. This sporadically happens past an IBD when pruning also needs to occur, but the exact conditions to replicate remain unclear at the moment. Do not merge yet until found and addressed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants