Skip to content

Conversation

@imikejackson
Copy link
Contributor

@imikejackson imikejackson commented Jan 28, 2026

Used Claude-Code (Sonnet 4.5) to reimplement this filter's algorithm to better optimize for out-of-core performance without hugely impacting the in-core performance.

Both out-of-core builds and in-core builds pass the same unit test indicating that for the given input data set we are producing the same results. This code should be actually verified with smaller, more concise data sets to completely validate the algorithm is producing what we expect it to produce.

Claude-Code generated more unit test files that better exercise the edge cases of the algorithm. The Small IN100 test case exercises the out-of-core performance.

@imikejackson imikejackson force-pushed the topic/fill_bad_data_ooc_optimization branch from c4d83cc to 602736b Compare January 28, 2026 23:58
@imikejackson imikejackson requested a review from nyoungbq January 29, 2026 15:07
Copy link
Contributor

@nyoungbq nyoungbq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, suggestion is mostly for the sake of cleanup. The Union-Find structure looks to be all good. From the few times I have used Union-Find, I am not sure whether compressing the paths should be done at every merge. If your doing frequent merges you are wasting cycles updating values that will be updated again if it gets merged again, rather than one pass through at flatten for future searches. I understand it cuts down the depths of the trees though, so perhaps that provides an optimization when searching the trees during construction. Also worth noting the types sometimes use our type declarations uint64 and others use the standard uint64_t, I would bring them all in line one way or the other. The rest looks good as far as I can tell.

@imikejackson imikejackson force-pushed the topic/fill_bad_data_ooc_optimization branch 5 times, most recently from 59b867c to 6286ca1 Compare January 30, 2026 22:48
These changes were implemented using the Claude Sonnet 4.5 model
Unit test now passes and matches that from DREAM3D v6.6
Documentation file is now updated.
Added function to UnitTestCommon to allow an RAII style for keeping your preferences intact.
@imikejackson imikejackson force-pushed the topic/fill_bad_data_ooc_optimization branch from 6286ca1 to 127b60d Compare February 2, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants