Skip to content

Optimize merge_contacts() for Databricks#195

Merged
fivetran-jamie merged 2 commits into
mainfrom
optimize-databricks-merge-ids
Feb 25, 2026
Merged

Optimize merge_contacts() for Databricks#195
fivetran-jamie merged 2 commits into
mainfrom
optimize-databricks-merge-ids

Conversation

@fivetran-jamie
Copy link
Copy Markdown
Collaborator

@fivetran-jamie fivetran-jamie commented Feb 24, 2026

PR Overview

Package version introduced in this PR:

  • 1.7.1

This PR addresses the following Issue/Feature(s):

  • GA-1016308

Summary of changes:

  • Adjusts merge_contacts() to use more performant lateral explode logic rather than a subquery

Submission Checklist

  • Alignment meeting with the reviewer (if needed)
    • Timeline and validation requirements discussed
  • Provide validation details: in jira
    • Validation Steps: Check for unintentional effects (e.g., add/run consistency & integrity tests)
    • Testing Instructions: Confirm the change addresses the issue(s)
    • Focus Areas: Complex logic or queries that need extra attention
  • Merge any relevant open PRs into this PR

Changelog

  • Draft changelog for PR
  • Final changelog for release review

@fivetran-jamie fivetran-jamie self-assigned this Feb 24, 2026
Comment on lines -270 to -273
ticket_data:
+column_types:
property_hs_pipeline: "{{ 'string' if target.type in ('bigquery', 'spark', 'databricks') else 'varchar' }}"
property_hs_pipeline_stage: "{{ 'string' if target.type in ('bigquery', 'spark', 'databricks') else 'varchar' }}"
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a duplicate config

@fivetran-jamie fivetran-jamie marked this pull request as ready for review February 24, 2026 20:43
Copy link
Copy Markdown
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-jamie Approved!

@fivetran-jamie fivetran-jamie added the docs:ready Triggers the docs generator workflow. label Feb 25, 2026
@fivetran-jamie fivetran-jamie merged commit 792b8ff into main Feb 25, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs:ready Triggers the docs generator workflow.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants