feat(reader): lazy DecimalByteParts decode#50
Merged
Conversation
The pre-existing DecimalBytePartsEncodingDecoder wrapped the decoded mantissa child in a GenericArray with zero buffers and one child. The decimal-from-child reassembly logic then lived inside GenericArray — buffers-vs-child shape dispatch, ptype switch on the child Array, mantissa BigInteger construction — fighting GenericArray's other role as the single-buffer fallback for vortex.decimal. Introduce LazyDecimalBytePartsArray (record, implements Array) that owns its own decimal reassembly. Its getDecimal(i) reads one cell from the mantissa child via DecimalBytePartsArrays.readMantissa (a package-private helper that dispatches Byte / Short / Int / Long Array, recurses through MaskedArray, throws on null cells) and combines the result with the parent DType.Decimal scale to produce a BigDecimal on demand. No buffer materialisation at construction time. DecimalBytePartsEncodingDecoder now returns LazyDecimalBytePartsArray directly. GenericArray.getDecimal is reduced to the single-buffer case only — the child-array branch is gone and the bounds / shape checks shrink accordingly. ScanIterator's truncate dispatch gains a LazyDecimalBytePartsArray case that recursively truncates the child and rebuilds the record at the clamped row count. VortexInspectorTui's data-cell renderer recognises both decimal Array flavours via a small tryDecimal helper that takes the getter as a LongFunction; the per-row formatting path stays identical. The two GenericArrayTest cases that exercised the child-array shape (getDecimal_childArrayShape_decodesViaMostSignificantPart and getDecimal_nullCellInMaskedChild_throws) move to the new LazyDecimalBytePartsArrayTest, joined by a widening case (i32 child under a decimal(15,2) parent) and the bounds / non-decimal-dtype guards. The writer's DecimalBytePartsEncodingEncoderTest round-trip now asserts the reassembled BigDecimal value rather than reaching into GenericArray.child(0). ./mvnw verify green (13 modules SUCCESS, integration suite 40s). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Convert
vortex.decimal_byte_partsdecoding from aGenericArraywrap (with the child-array reassembly hard-coded intoGenericArray.getDecimal) into a dedicated lazyLazyDecimalBytePartsArrayrecord that owns its own reassembly.Background
The pre-existing decoder wrapped the mantissa child in
GenericArray(dtype, len, [], [mspChild]). The decimal-from-child path then lived insideGenericArray.getDecimal— buffers-vs-child shape dispatch, ptype switch on the child,BigIntegerconstruction — fightingGenericArray's other role as the single-buffer fallback forvortex.decimal.Changes
LazyDecimalBytePartsArray(record, implementsArray) holds parentDType.Decimal, length, and the mantissa child.getDecimal(i)reads one cell from the child via a small package-private helper and combines it with the dtype scale to produce aBigDecimalon demand. No buffer materialisation at construction time.DecimalBytePartsArrays(package-private) centralises the per-row mantissa read — dispatchesByte / Short / Int / Long Array, recurses throughMaskedArray, throws on null cells.DecimalBytePartsEncodingDecodernow returnsLazyDecimalBytePartsArraydirectly.GenericArray.getDecimalis reduced to the single-buffer layout only; the child-array branch and helper are gone.ScanIteratortruncate dispatch gains aLazyDecimalBytePartsArraycase that recursively truncates the child and rebuilds the record at the clamped row count.VortexInspectorTuidata-cell renderer recognises both decimal flavours via a smalltryDecimalhelper taking the getter as aLongFunction.Tests
GenericArrayTestcases that exercised the child-array shape moved to the newLazyDecimalBytePartsArrayTest, joined by a widening case (i32 child under decimal(15,2)) and the bounds / non-decimal-dtype guards.DecimalBytePartsEncodingEncoderTestround-trip now asserts the reassembledBigDecimalvalue rather than reaching intoGenericArray.child(0).Test plan
./mvnw verify— 13 modules SUCCESS, integration suite 40s green.main(no{@code}left in new Javadoc).🤖 Generated with Claude Code