feat: add feature for switching to writeColumnBatches endpoint#227
Open
alxhill wants to merge 3 commits into
Open
feat: add feature for switching to writeColumnBatches endpoint#227alxhill wants to merge 3 commits into
alxhill wants to merge 3 commits into
Conversation
Adds the `columnar` cargo feature that switches from writeNominalBatches (row-oriented, dataSourceRid in URL path) to writeNominalColumnarBatches (columnar, dataSourceRid in proto body). Introduces a `StreamWriteRequest` type alias in client.rs that switches between WriteRequestNominal and WriteBatchesRequest based on the feature; this unifies trait signatures and consumer-wrapper types across both feature states. Implementation details (AvroFileConsumer body, SeriesBuffer::take, batch_processor request construction, encoding path, and tests) are feature-gated in-line with `#[cfg(feature = "columnar")]`. First pass — 55 cfg annotations across 5 files; follow-up will move the divergent code into dedicated modules so each side sits behind a single cfg boundary. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Moves all feature-specific helpers (encoding, avro serialization, request construction, log summaries, format-specific tests) from per-callsite cfg blocks in shared files into two dedicated sibling modules: src/row.rs — compiled when `columnar` is disabled src/columnar.rs — compiled when `columnar` is enabled `lib.rs` aliases the active module as `crate::format`, so all other files (`consumer.rs`, `stream.rs`, `listener.rs`) call `crate::format::*` without any cfg annotations. Each format module exposes the same internal API: build_write_request — turn drained SeriesBuffer into a StreamWriteRequest encode_for_core — serialize + wrap in an HTTP WriteRequest avro_append — write a request to an avro file request_summary — short human-readable payload summary test_support::* — helpers the stream tests use to assert on payloads Removes the duplicated assertion blocks in `lib.rs` stream tests in favor of `format::test_support::count_points_by_channel` and `first_request_first_double_count`; assertions now read the aggregate count correctly (5 × 1000 points, not the accidental last-series 1000). Result: 55 cfg annotations → 8, concentrated at the two boundary points (module decls in lib.rs + the StreamWriteRequest alias in client.rs). consumer.rs, stream.rs, listener.rs carry zero cfg. Shared trait signatures and consumer-wrapper types (Fallback/Dual/Listening) remain unified via the StreamWriteRequest alias. Both feature states compile clean under clippy -D warnings and all 6 tests pass in each. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
building on top of #210, add the use of the new endpoint as an opt-in feature for easier testing rather than a hard switchover