Skip to content

Icebird WRITER#14

Merged
platypii merged 1 commit intomasterfrom
icebird-writer
Apr 28, 2026
Merged

Icebird WRITER#14
platypii merged 1 commit intomasterfrom
icebird-writer

Conversation

@platypii
Copy link
Copy Markdown
Contributor

Read path

  • Iceberg v3 reads: row lineage (_row_id, _last_updated_sequence_number), v3 types (variant, geometry, geography), Puffin deletion vectors, equality delete matching, delete partition scope
  • Hidden + identity partition support, manifest-list partition pruning, parallelized data-file reads
  • Gzip metadata JSON (.gz.metadata.json, metadata.json.gz)

Write path (experimental, v2 + v3 deletion vectors)

  • icebergCreateTable, icebergDropTable, icebergAppend, icebergDelete, icebergSetRef, icebergExpireSnapshots
  • Generic Catalog abstraction — same calls work against fileCatalog or REST
  • Position deletes as Parquet or Puffin deletion vectors; equality delete manifests
  • Append/staged-commit flow with per-column stats, sort orders, partition spec evolution, fixed/decimal + geospatial stats, lower/upper bound truncation
  • Honors format-version, write.format.default, write.metadata.previous-versions-max, write-default, compressor options
  • Schema/spec updates: add-schema, set-current-schema, add-spec/set-default-spec, set-properties/remove-properties, full assert-* requirement checks, V3-only-type validation
  • Async hyparquet writer, in-memory end-to-end test harness

REST catalog

  • Connect, list/load tables, create/drop/rename table, create/delete namespace, register table, update table, load credentials

I/O

  • New Resolver/Lister abstraction (pluggable storage); urlResolver ships writer/deleter for HTTP PUT/DELETE
  • Auth now flows through requestInit on the resolver/lister rather than per-call (breaking)

@platypii platypii merged commit c0d1835 into master Apr 28, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant