VV: EbsdLib 3.0.0 + V&V of 6 Filters by imikejackson · Pull Request #1631 · BlueQuartzSoftware/simplnx

imikejackson · 2026-06-04T22:29:16Z

No description provided.

Adds a "Color Key" ChoicesParameter to ComputeIPFColorsFilter and ComputeFaceIPFColoringFilter so pipelines can select between the three EbsdLib IPF coloring schemes per filter invocation: 0 = TSL (EDAX/OIM Analysis default -- preserved as default) 1 = PUCM (Patala perceptually-uniform color map) 2 = Nolze-Hielscher (MTEX HSV-style coloring) The choice index is mapped to ebsdlib::ColorKeyKind in executeImpl and threaded through the algorithm's InputValues struct + Impl-class constructor to the generateIPFColor(... kind) call site. Each LaueOps subclass already owns its own per-class TSL/PUCM/NH singletons sized to its rotation point group / fundamental sector, so no per-pixel allocation is added. parametersVersion is bumped to 2 on both filters. Also fixes WritePoleFigure.cpp's templated `createIntensityPoleFigures` which previously called the 4-arg generateSphereCoordsFromEulers; the new EbsdLib API requires an explicit HexConvention. Passing config.hexConvention threads the existing PoleFigureConfiguration_t field down to the LaueOps call. Tests: - Existing TSL-regression tests stay green (default kind = TSL preserves byte-for-byte exemplar match). - New "ColorKey choice reaches algorithm" plumbing test per filter: runs the filter three times with k_ColorKey_Key = 0/1/2 into uniquely-named output arrays and asserts non-TSL kinds produce a different output than TSL. Catches any future off-by-one in the executeImpl switch without duplicating EbsdLib's per-Laue-class color correctness coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Pole-figure positions and corner labels for hex/trig phases differ by a 30° rotation about the c-axis between the two Cartesian basis conventions in EbsdLib (X||a vs X||a*). Pre-this-commit, the filter silently used PoleFigureConfiguration_t's default (X||a*) -- pipelines operating on EDAX/TSL/OIM Analysis data had no way to pick the convention that matches their input stack. Adds a "Hex/Trig Cartesian Basis Convention" ChoicesParameter: 0 = X parallel to a (EDAX/TSL/OIM Analysis -- new default) 1 = X parallel to a* (MTEX / Oxford / AZtec) Default is X||a because every released DREAM.3D / DREAM3DNX / SIMPL / SIMPLNX file stores hex/trig EulerAngles in this form by codebase guarantee (see ebsdlib::HexConvention doc comment in EbsdLibConstants.h). Cubic, tetragonal, orthorhombic, monoclinic, and triclinic phases ignore the parameter -- existing all-cubic test fixtures are unaffected by the default flip. The choice index is mapped to ebsdlib::HexConvention in executeImpl and threaded through WritePoleFigureInputValues to PoleFigureConfiguration_t::hexConvention, which the EbsdLib generateSphereCoordsFromEulers call already honors. parametersVersion bumps to 2. Tests: - New "HexConvention choice reaches algorithm" plumbing test: runs the filter twice on the cubic exemplar with k_HexConvention_Key = 0 and = 1, captures the intensity arrays, and asserts they're identical (cubic is convention-invariant). Catches parameter wiring failures without depending on hex/trig exemplar data. Note: The 4 existing WritePoleFigureFilter-{Discrete,Color}[-Masked] exemplar-pixel-compare tests fail with mismatches at byte index 972772 of the rendered image. Those failures predate this commit (verified by stashing the filter changes; same 4 tests fail at the same byte). The PoleFigure_Exemplars_v5.tar.gz baseline appears to have drifted from the current EbsdLib pole figure rendering output -- regenerating the exemplar as _v6 is the right fix and is out of scope here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…st updates

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

…entations Compute Kernel Average Misorientations fully V&V'ed Summary: - Confirmed no SIMPLNX-side bugs (legacy D2 inner-x-loop typo was corrected at port time); - documented 2 deviations from DREAM3D 6.5.171 (D1 EbsdLib 2.4.1 precision, D2 legacy kernel-bound typo); - retired 1 test (circular-oracle exemplar consumer regenerated from pre-EbsdLib-2.4.1 SIMPLNX output); - unit tests replaced with 5 inlined *Class 1 (Analytical) + Class 4 (Invariant)* test fixtures; - added 3 V&V source-tree deliverables (report, deviations, provenance); - fixed user-facing doc (pipeline-name typo, dropped orphan MassifPipeline reference, added EBSD_File_Processing examples). VV: Compute Feature Neighbor Misorientations Summary: - Found and fixed 1 bug (divisor clobbered inside inner j-loop of algorithm.cpp); - documented 2 deviations from DREAM3D 6.5.171 (D1 divisor bug, D2 EbsdLib 2.4.1 precision); - retired 2 tests (circular-oracle exemplar consumer + UNIMPLEMENTED stub); - unit tests replaced with 4 inlined *Class 1 (Analytical) + Class 4 (Invariant)* test fixtures; - added 3 V&V source-tree deliverables (report, deviations, provenance); - fixed pipeline-name typo in user-facing doc. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

…n V&V'ed BadDataNeighborOrientationCheck: - Confirmed no SIMPLNX-side bugs (D1 was landed pre-V&V in PR #1499; D2 was SIMPLNX-correct from rewrite onward); - documented 2 deviations from DREAM3D 6.5.171 (D1 iterative-decay loop bound `>` should be `>=`, D2 stale-w threshold check leaks across different-phase neighbors); - retired 1 test (legacy 6_6 exemplar-comparison TEST_CASE — replaced with the engineer's hand-derived 27-fixture dataset); - unit tests replaced with 27 hand-derived Class 1 (Analytical) base fixtures + 18 Class 4 (Invariant) Invariants-Sweep DYNAMIC_SECTIONs + 1 Class 4 Idempotence test + 1 inlined 2D Image fixture (31 TEST_CASEs / 49 ctest entries, 100% pass); - added 3 V&V source-tree deliverables (report, deviations, provenance); - restored bad_data_neighbor_orientation_check_v2.tar.gz archive download in test/CMakeLists.txt. Compute Feature Face Misorientation: - Confirmed no SIMPLNX-side bugs (D3 fix corrects a legacy 6.5.171 bug — SIMPLNX has been correct since the rewrite); - documented 4 deviations from DREAM3D 6.5.171 (D1 9 additional Laue classes now supported, D2 1-component magnitude output vs legacy 3-component axis·angle, D3 NaN sentinel for unprocessed faces vs legacy implicit 0, D4 EbsdLib 2.4.1 CubicOps precision improvement); - retired 1 test ("Invalid filter execution" — preflight-failure paths made unreachable by D3's NaN-on-invalid-face semantics); - unit tests replaced with 1 hand-built Class 1 (Analytical) 37-fixture dataset (30 normal cases across all 11 EbsdLib Laue classes × 3 pure-φ1 boundaries + 4 edge cases + 3 Trigonal_High cases) + 1 SIMPL 6.4/6.5 backwards-compat; - added 3 V&V source-tree deliverables (report, deviations, provenance); - fixed user-facing doc.

Summary: - Found and fixed 1 bug (divisor clobbered inside inner j-loop of algorithm.cpp, sibling of F#2's D1 — production-relevant via shipping EBSD_Hexagonal_Data_Analysis.d3dpipeline which runs with find_avg_misals=true); - documented 5 deviations from DREAM3D 6.5.171 (D1 divisor bug, D2 avg-array fillValue uncertainty (latent, confirmed dormant on current backend), D4 EbsdLib quat-matrix swap precision drift, D5 PR #1438 preflight-banner UX downgrade, D6 Hexagonal_Low support gap); - retired 1 test (hex-only exemplar consumer — could not trigger the divisor bug; removed its download_test_data line from test/CMakeLists.txt); - unit tests replaced with 4 inlined Class 1 (Analytical) + Class 4 (Invariant) test fixtures (incl. a 10x10x1 6-feature realistic microstructure with 3 bug-exposing per-feature configurations); - added 3 V&V source-tree deliverables (report, deviations, provenance); - performed empirical A/B against 3 binaries (DREAM3D 6.5.171 official, 6.5.172 with D1+D4+D6 backports, SIMPLNX) and confirmed byte-for-byte match between 6.5.172 (post-backport) and SIMPLNX — 24/24 values bit-identical. Artifacts at /Users/mjackson/Desktop/F6_AB_Test/; - fixed Hexagonal_Low note in user-facing doc. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

Summary: - Confirmed no SIMPLNX-side bugs (clean Port; algorithm verified correct as-is — only cleanup applied: [SimplnxReview]→[SimplnxCore] test-tag fix on 7 tests + code-comment cleanup in the algorithm + filter source); - documented 0 deviations from the pre-SIMPLNX legacy (legacy is the un-merged `tuks188/DREAM3D` `feature/770_Grouping_Density` branch, not 6.5.171 shipped); empirical A/B against a locally-rebuilt legacy binary produced BIT-IDENTICAL output across all 4 (UseNonContiguous, FindCheckedFeatures) configurations of both GroupingDensities and CheckedFeatures; - retired 5 tests (4 redundant execution tests + 1 v1-exemplar consumer); v1 archive `compute_grouping_densities.tar.gz` removed from test/CMakeLists.txt; - unit tests replaced with 1 DYNAMIC_SECTION 4-config exemplar A/B test (Class 1 Analytical + Class 4 Invariant oracles, hand-derivation in the V&V report) + 1 Class 4 empty-parent sentinel test (-1.0f sentinel) + 1 preflight-error -15672 test (gap closed); 3 existing preflight error tests + 1 SIMPL backwards-compat test retained — 7 TEST_CASEs total; - added 2 V&V source-tree deliverables (report, deviations); provenance materially captured in the v2 archive's inline ReadMe + comparison report rather than a separate vv/provenance/ file (the v2 tarball contains input + all 4 legacy and SIMPLNX outputs + comparison script + sign-off ReadMe); - published new `compute_grouping_densities_v2.tar.gz` to the GitHub Data_Archive release (SHA512 wired into test/CMakeLists.txt) with hand-review sign-off ReadMe; - updated user-facing doc (Required Input Sources w/ MyST cross-links, explicit volume units, tie-break behavior, italicized -1.0 sentinel) and added a new ComputeGroupingDensity_Infographic (PNG + SVG); small core-DataStructure header cleanup (IListStore/ListStore/EmptyListStore/AbstractListStore — `uint64`→`usize` consistency + virtual `size()`). Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

Summary: - Confirmed no SIMPLNX-side bugs (algorithm verified correct as-is); applied 2 algorithm refinements during the Phase-7 review (counter==0 → NaN at finalize + final per-feature normalize) — these are intentional design alignments that surface as D1 and D2 deviations vs DREAM3D 6.5.171 but match the intended algorithm and are bit-identical to the 6.5.172 backport; - documented 2 deviations from DREAM3D 6.5.171 (D1 counter==0 → NaN at finalize replaces legacy's (0,0,1) rescue at F0/F5/F6; D2 precision-sensitive direction + unit-vector vs unnormalized magnitude at F7 antipodal-flip cancellation boundary); - retired 1 test (legacy-by-reputation exemplar consumer of 7_2_AvgCAxis.tar.gz — confirmed circular oracle per policy line 33: reference values were produced by a "special build of DREAM3D 6.6.379 with micro-texture bug fixes," not by an independent oracle); - unit tests replaced with 3 tests: 1 Class 1 (Analytical) valid-execution exemplar against a hand-built 11-cell/8-feature dataset + 1 negative all-non-hex error test + 1 SIMPL 6.4/6.5 backwards-compat DYNAMIC_SECTION — covers 8/12 enumerated code paths (gaps: all-hex-ensemble preflight, background-voxel skip, 2 cancel-check paths); - added 3 V&V source-tree deliverables (report, deviations, provenance — provenance file attributes the dataset design to MAJ with subsequent design changes and Python generation-script integration by Claude Opus 4.7, signed off by MAJ); - published new compute_avg_c_axis.tar.gz (replaces retired 7_2_AvgCAxis.tar.gz) with hand-built Class 1 dataset + Class 4 (Invariant) ||AvgCAxes|| == 1.0 unit-vector contract for F7; empirical three-way A/B (SIMPLNX vs 6.5.171 official vs 6.5.172 backport) confirmed SIMPLNX bit-identical to 6.5.172 across all 8 features, conclusively isolating D1 and D2 root causes; - updated user-facing doc with new ComputeAvgCAxes_HexagonalCAxis figure (SVG + PNG) explaining the c-axis-in-sample-frame convention. Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

nyoungbq

Approved, I didn't find any issues. The V&V stuff does feel overly wordy, but I think the information being conveyed still comes across so I am approving. On the C-Axis V&V, since you asked specifically, the python script processing of the data to make it suitable for both version of SIMPL came across fine. I understood the logic flow, so I would leave it.

JDuffeyBQ

There's a commit titled "BUG: WriteImageFilter preflight did not validate tuple dimensions" that has no changes to WriteImageFilter?

JDuffeyBQ · 2026-06-05T18:42:59Z

+  auto differs = [](const UInt8Array& a, const UInt8Array& b) {
+    const size_t size = a.getSize();
+    for(size_t i = 0; i < size; ++i)
+    {
+      if(a[i] != b[i])
+      {
+        return true;
+      }
+    }
+    return false;
+  };


This is just !std::equal(a, b)

JDuffeyBQ · 2026-06-05T18:43:26Z

+  auto differs = [](const UInt8Array& a, const UInt8Array& b) {
+    const size_t size = a.getSize();
+    for(size_t i = 0; i < size; ++i)
+    {
+      if(a[i] != b[i])
+      {
+        return true;
+      }
+    }
+    return false;
+  };


This is just !std::equal(a, b)

JDuffeyBQ · 2026-06-05T18:44:45Z

+  default:
+    inputValues.colorKey = ebsdlib::ColorKeyKind::TSL;
+    break;


Rather than default I think it should be the exact value. A value out of range of an enum is an error. And this is also done in the filter as well so it could be a function.

JDuffeyBQ · 2026-06-05T18:45:46Z

+  default:
+    inputValues.ColorKey = ebsdlib::ColorKeyKind::TSL;
+    break;
+  }


See comment in algorithm class

JDuffeyBQ · 2026-06-05T18:47:26Z

+  default:
+    inputValues.HexConvention = ebsdlib::HexConvention::XParallelA;
+    break;


Similar to other enum comment.

JDuffeyBQ · 2026-06-05T19:06:19Z

+      const std::string errorMsg = fmt::format("Idempotence violated at index {}. Run1: {} | Run2: {}", i, maskAfterRun1[i], store.getValue(i));
+      CAPTURE(errorMsg);


This could use INFO() instead.

JDuffeyBQ · 2026-06-05T19:07:20Z

+      const std::string errorMsg = fmt::format("2D fixture: values diverged at index {}. Expected: {} | Actual: {}", i, expectedMask[i], maskStore.getValue(i));
+      CAPTURE(errorMsg);


This could use INFO() instead.

JDuffeyBQ · 2026-06-05T19:09:12Z

+    // resulting divisor only reflected the LAST neighbor's match/mismatch state, producing wrong
+    // per-feature averages whenever neighbors had mixed phases. Fixed 2026-06-04 during V&V cycle
+    // (sibling of the same divisor bug fixed in ComputeFeatureNeighborMisorientations on 2026-06-02).
+    hexNeighborListSize = currentNeighborList.size();


Definition here could also be moved into loop.

JDuffeyBQ · 2026-06-05T19:09:25Z

+// {numFeatures}; ensemble-level arrays (CrystalStructures) are sized {numCrystalStructures}.
+// Defaults: every cell assigned to feature 1 / phase 1; every feature phase 0 (caller to set);
+// identity quats; empty neighbor lists; CrystalStructures[0] = sentinel 999.
+inline ToyData CreateScaffold(usize nX, usize nY, usize nZ, usize numFeatures, usize numCrystalStructures)


inline issue

JDuffeyBQ · 2026-06-05T19:12:05Z

+   * @return usize The number of lists
   */
-  virtual uint64 getNumberOfLists() const = 0;
+  virtual usize getNumberOfLists() const = 0;


Is there a reason for changing this type?

Because it is a USIZE, and not a uint64.

I was mostly just wondering if there was a reason it had been uint64 previously.

imikejackson requested a review from JDuffeyBQ June 4, 2026 22:29

github-actions Bot reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Plugins/SimplnxCore/src/SimplnxCore/Filters/CreateColorMapFilter.cpp Outdated

imikejackson force-pushed the topic/ebsdlib_v3_updates branch from da22124 to 92f0f78 Compare June 5, 2026 01:19

imikejackson and others added 6 commits June 4, 2026 21:19

ENH: Pole Figure output is now a PNG file instead of a TIFF file.

1b0dd05

BUG: WriteImageFilter preflight did not validate tuple dimensions

c43ec2f

BUG: WritePoleFigure filter bug fixes for Hex convention and unit te…

909feac

…st updates

VER: Update to EbsdLib version 3.0.0

d417422

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

imikejackson force-pushed the topic/ebsdlib_v3_updates branch from 92f0f78 to d417422 Compare June 5, 2026 01:20

imikejackson mentioned this pull request Jun 5, 2026

VV: EbsdLib 3.0.0 + V&V of 6 Filters #1633

Closed

imikejackson changed the title ~~VER: Update to EbsdLib version 3.0.0~~ VV: EbsdLib 3.0.0 + V&V of 6 Filters Jun 5, 2026

imikejackson force-pushed the topic/ebsdlib_v3_updates branch from d4393a1 to c5120ff Compare June 5, 2026 15:29

imikejackson added 5 commits June 5, 2026 11:42

Marking V&V filter's status as complete

cf15afd

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>

imikejackson requested a review from nyoungbq June 5, 2026 16:14

nyoungbq approved these changes Jun 5, 2026

View reviewed changes

COMP: Change to the real EbsdLib 3.0.0 release.

fbc03e2

JDuffeyBQ requested changes Jun 5, 2026

View reviewed changes

		const std::string errorMsg = fmt::format("Idempotence violated at index {}. Run1: {} \| Run2: {}", i, maskAfterRun1[i], store.getValue(i));
		CAPTURE(errorMsg);

Conversation

imikejackson commented Jun 4, 2026

Uh oh!

Uh oh!

nyoungbq left a comment

Choose a reason for hiding this comment

Uh oh!

JDuffeyBQ left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants