Enhanced Data Type Support and CDC Configuration for Postgres

[vikaxsh]

Problem

Postgres data type handling is currently fragmented, with many complex types falling back to string serialization. Additionally, CDC configuration options are too restrictive for advanced users.

• Complex types (json, jsonb, uuid, etc.) often fall back to string serialization.
• Manual OID mapping is incomplete and hard to maintain.
• Plugin configuration cannot accept custom replication parameters.

Solution

• Refactor type mapping to use structured types instead of strings.
• Use pgtype for automatic OID resolution.
• Expose plugin arguments in CDC driver configuration.

[DhineshPonnarasan]

Hi, Shall I work on this?

[nayanj98]

@DhineshPonnarasan assigning the issue to you. Please make sure you while raising a PR you also upload a video of showing the source table records running a sync with OLake and then showing the same records on Iceberg itself. Without this video we won’t start reviewing your PR

[DhineshPonnarasan]

Thanks for assigning the issue.

Understood on the requirements. I’ll:

• Refactor Postgres type handling to use pgtype for structured decoding
• Reduce reliance on manual OID mapping
• Expose configurable replication plugin arguments in the CDC driver
• Validate the changes end-to-end using a real Postgres → OLake → Iceberg sync

I’ll also attach a short demo video with the PR showing:

1. Source Postgres table records
2. OLake CDC sync execution
3. The corresponding records written to Iceberg

I’ll keep the changes focused and backward-compatible. I’ll reach out on the issue if I have any questions during implementation.

[Akshay-datazip]

@DhineshPonnarasan is there any progress on this, or are you facing any issues? You can go ahead and simply reach out to me without any hesitation.

[DhineshPonnarasan]

Okay sure @nayanj98 .
Thanks for the headups

[DhineshPonnarasan]

Hi @nayanj98 ,
Here is the PR: #833

Can you verify and please let me know if there’s anything I need to record and share for verification.
Thanks.

[nayanj98]

@DhineshPonnarasan Have assigned a reviewer to your PR

[DhineshPonnarasan]

Thanks @nayanj98 , for assigning a reviewer.