[Project] Refunds, returns, discounts daily metric updates#147
Conversation
…roject/rest-refunds-returns-discounts-daily-aggregates
…dels, update seed data, etc.
…' of https://github.com/fivetran/dbt_shopify into project/rest-refunds-returns-discounts-daily-aggregates
fivetran-joemarkiewicz
left a comment
There was a problem hiding this comment.
Still reviewing, but a few small change requests before completing the review.
fivetran-avinash
left a comment
There was a problem hiding this comment.
@fivetran-joemarkiewicz First review note changes applied.
fivetran-joemarkiewicz
left a comment
There was a problem hiding this comment.
@fivetran-avinash thanks for working through these updates. A few change requests, questions, and suggestions before approval.
| coalesce(daily_orders.gross_sales, 0) as gross_sales, | ||
| coalesce(daily_orders.discounts, 0) as discounts, | ||
| coalesce(daily_orders.returns, 0) as returns, | ||
| coalesce(daily_orders.net_sales, 0) as net_sales |
There was a problem hiding this comment.
These fields need to be documented in the yml description for this model.
| coalesce(order_lines.gross_sales, 0) as gross_sales, | ||
| coalesce(order_lines.discount_allocation_amount, 0) as discounts, | ||
| coalesce(refund_aggregates.refund_subtotal_non_gift_card, 0) - coalesce(order_adjustments_aggregates.refund_discrepancy_amount, 0) as returns, | ||
| coalesce(order_lines.gross_sales, 0) - coalesce(order_lines.discount_allocation_amount, 0) - (coalesce(refund_aggregates.refund_subtotal_non_gift_card, 0) - coalesce(order_adjustments_aggregates.refund_discrepancy_amount, 0)) as net_sales, |
There was a problem hiding this comment.
These fields need to be documented in the yml description for this model.
Batch updates applied. Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
fivetran-avinash
left a comment
There was a problem hiding this comment.
@fivetran-joemarkiewicz Should be ready for re-review on Monday (BK is still running but passed before implementing minor changes.
fivetran-joemarkiewicz
left a comment
There was a problem hiding this comment.
Approved with a few small changes before release review
| | Data Model(s) | Change type | Old | New | Notes | | ||
| | ------------- | ----------- | --- | --- | ----- | | ||
| | [shopify__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refund_lines) or<br>[shopify_gql__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refund_lines) | New model | | | New final model at the refund line item grain. Exposes per-line refund financials, product context, and restock classification. | | ||
| | [shopify__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refunds) or<br>[shopify_gql__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refunds) | New model | - | - | New final model at the refund transaction grain. Aggregates line item financials and includes discrepancy adjustments for accurate reconciliation. | |
There was a problem hiding this comment.
The same suggestion from before wasn't applied to the other rows in this schema/data change section. Please remove the - from each empty field.
| | [shopify__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refunds) or<br>[shopify_gql__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refunds) | New model | - | - | New final model at the refund transaction grain. Aggregates line item financials and includes discrepancy adjustments for accurate reconciliation. | | |
| | [shopify__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refunds) or<br>[shopify_gql__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refunds) | New model | | | New final model at the refund transaction grain. Aggregates line item financials and includes discrepancy adjustments for accurate reconciliation. | |
| | [shopify__discounts](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__discounts) or<br>[shopify_gql__discounts](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__discounts) | Discount code performance with redemption rates, revenue impact, and customer usage patterns for promotional campaign analysis. <br><br>**Example Analytics Questions:**<br>• Which discount codes have the highest redemption rates and total revenue impact?<br>• What is the average order value for customers using percentage vs fixed amount discounts? | | ||
| | [shopify__inventory_levels](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__inventory_levels) or<br>[shopify_gql__inventory_levels](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__inventory_levels) | Product variant inventory tracking by location with sales performance, stock levels, and fulfillment metrics for supply chain optimization. Includes `PRODUCT_VARIANT` metafields if enabled. <br><br>**Example Analytics Questions:**<br>• Which product variants have the lowest inventory levels relative to their sales velocity?<br>• What is the total inventory value by location and which locations are most profitable? | | ||
| | [shopify__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refunds) or<br>[shopify_gql__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refunds) | One record per refund transaction, with line item financials aggregated to the refund level. Use this model when you need refund-level totals (e.g., total amount refunded, total items restocked, discrepancy adjustments) or when joining refund data to order-level context. You cannot derive per-line product detail or per-line restock classification from this model alone. <br><br>**Example Analytics Questions:**<br>• What is the total refund value per order and how does it compare to the original order value?<br>• What is the average time between order creation and refund processing? | | ||
| | [shopify__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refund_lines) or<br>[shopify_gql__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refund_lines) | One record per refund line item — the most granular refund grain. Use this model when you need to analyze refunds at the product or variant level, including per-line financials, restock classification, and product context. You cannot derive this detail from the refunds model without re-joining to source tables. <br><br>**Example Analytics Questions:**<br>• Which product variants have the highest refund rates and what restock types are most common?<br>• What is the refund amount breakdown by product category and restock classification? | |
There was a problem hiding this comment.
| | [shopify__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refund_lines) or<br>[shopify_gql__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refund_lines) | One record per refund line item — the most granular refund grain. Use this model when you need to analyze refunds at the product or variant level, including per-line financials, restock classification, and product context. You cannot derive this detail from the refunds model without re-joining to source tables. <br><br>**Example Analytics Questions:**<br>• Which product variants have the highest refund rates and what restock types are most common?<br>• What is the refund amount breakdown by product category and restock classification? | | |
| | [shopify__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refund_lines) or<br>[shopify_gql__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refund_lines) | One record per refund line item, the most granular refund grain. Use this model when you need to analyze refunds at the product or variant level, including per-line financials, restock classification, and product context. You cannot derive this detail from the refunds model without re-joining to source tables. <br><br>**Example Analytics Questions:**<br>• Which product variants have the highest refund rates and what restock types are most common?<br>• What is the refund amount breakdown by product category and restock classification? | |
| dbt run --vars '{shopify_schema: shopify_integration_tests_sqlw}' --target "$db" --full-refresh | ||
| dbt test --vars '{shopify_schema: shopify_integration_tests_sqlw}' --target "$db" | ||
| dbt run --vars '{shopify_schema: shopify_integration_tests_sqlw, shopify_timezone: "America/New_York", shopify_using_fulfillment_event: true, shopify_gql_using_fulfillment_event: true, shopify_using_all_metafields: true, shopify__calendar_start_date: '2020-01-01', shopify_using_abandoned_checkout: false, shopify_gql_using_abandoned_checkout: false, shopify_using_metafield: false, shopify_gql_using_metafield: false, shopify_using_discount_code_app: true, shopify_gql_using_discount_code_app: true, shopify_using_product_variant_media: true, shopify_gql_product_variant_media: true, shopify_gql_using_collection_rule: true, shopify_gql_using_customer_visit: false}' --target "$db" --full-refresh | ||
| dbt run --vars '{shopify_schema: shopify_integration_tests_sqlw, shopify_timezone: "America/New_York", shopify_using_fulfillment_event: true, shopify_gql_using_fulfillment_event: true, shopify_using_all_metafields: true, shopify__calendar_start_date: '2020-01-01', shopify_using_abandoned_checkout: false, shopify_gql_using_abandoned_checkout: false, shopify_using_metafield: false, shopify_gql_using_metafield: false, shopify_using_discount_code_app: true, shopify_gql_using_discount_code_app: true, shopify_using_product_variant_media: true, shopify_gql_product_variant_media: true, shopify_gql_using_collection_rule: true, shopify_gql_using_customer_visit: false, shopify_gql_using_return: false}' --target "$db" --full-refresh |
There was a problem hiding this comment.
This variable is false by default. Let's test the true state here instead.
| dbt run --vars '{shopify_schema: shopify_integration_tests_sqlw, shopify_timezone: "America/New_York", shopify_using_fulfillment_event: true, shopify_gql_using_fulfillment_event: true, shopify_using_all_metafields: true, shopify__calendar_start_date: '2020-01-01', shopify_using_abandoned_checkout: false, shopify_gql_using_abandoned_checkout: false, shopify_using_metafield: false, shopify_gql_using_metafield: false, shopify_using_discount_code_app: true, shopify_gql_using_discount_code_app: true, shopify_using_product_variant_media: true, shopify_gql_product_variant_media: true, shopify_gql_using_collection_rule: true, shopify_gql_using_customer_visit: false, shopify_gql_using_return: false}' --target "$db" --full-refresh | |
| dbt run --vars '{shopify_schema: shopify_integration_tests_sqlw, shopify_timezone: "America/New_York", shopify_using_fulfillment_event: true, shopify_gql_using_fulfillment_event: true, shopify_using_all_metafields: true, shopify__calendar_start_date: '2020-01-01', shopify_using_abandoned_checkout: false, shopify_gql_using_abandoned_checkout: false, shopify_using_metafield: false, shopify_gql_using_metafield: false, shopify_using_discount_code_app: true, shopify_gql_using_discount_code_app: true, shopify_using_product_variant_media: true, shopify_gql_product_variant_media: true, shopify_gql_using_collection_rule: true, shopify_gql_using_customer_visit: false, shopify_gql_using_return: true}' --target "$db" --full-refresh |
…' of https://github.com/fivetran/dbt_shopify into project/rest-refunds-returns-discounts-daily-aggregates
There was a problem hiding this comment.
Looking pretty good! Left some comments, but I think this will require some more changes as well:
Because there are new fields being added downstream to the following models that contain metafields, I think you need to adjust the counts in this macro:
- daily_shop model
- orders model
| sum(coalesce(discount_allocation_agg.discount_allocation_amount, 0)) as product_total_discount, | ||
| sum(order_lines.order_line_tax) as product_total_tax, | ||
| avg(order_lines.quantity) as avg_quantity_per_order_line, | ||
| avg(order_lines.total_discount) as product_avg_discount_per_order_line, | ||
| avg(coalesce(discount_allocation_agg.discount_allocation_amount, 0)) as product_avg_discount_per_order_line, |
There was a problem hiding this comment.
Might wanna document these coalesces
There was a problem hiding this comment.
Added a note in the code.
There was a problem hiding this comment.
Ah I mean in the CHANGELOG, as some values may be changing from null to 0. Could go in the Under the Hood section
Not of the utmost importance but could be good to have there
There was a problem hiding this comment.
Makes sense. Updated. Just added to the relevant shopify_*_products row since it is technically a data change.
| | ------------- | ----------- | --- | --- | ----- | | ||
| | [shopify__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refund_lines) or<br>[shopify_gql__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refund_lines) | New model | | | New final model at the refund line item grain. Exposes per-line refund financials, product context, and restock classification. | | ||
| | [shopify__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refunds) or<br>[shopify_gql__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refunds) | New model | | | New final model at the refund transaction grain. Aggregates line item financials and includes discrepancy adjustments for accurate reconciliation. | | ||
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | Data change | `refund_subtotal`, `refund_total_tax`, and `count_orders_with_refunds` bucketed by order creation date | Now bucketed by refund creation date | **Possible breaking change** New refund metrics now likely to change values. Aligns with Shopify Finance report behavior, which attributes refunds to the date the refund was issued. | |
There was a problem hiding this comment.
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | Data change | `refund_subtotal`, `refund_total_tax`, and `count_orders_with_refunds` bucketed by order creation date | Now bucketed by refund creation date | **Possible breaking change** New refund metrics now likely to change values. Aligns with Shopify Finance report behavior, which attributes refunds to the date the refund was issued. | | |
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | Data change | `refund_subtotal`, `refund_total_tax`, and `count_orders_with_refunds` bucketed by order creation date | Now bucketed by refund creation date | **Possible breaking change:** New refund metrics now likely to change values. Aligns with Shopify Finance report behavior, which attributes refunds to the date the refund was issued. | |
| | [shopify__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refund_lines) or<br>[shopify_gql__refund_lines](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refund_lines) | New model | | | New final model at the refund line item grain. Exposes per-line refund financials, product context, and restock classification. | | ||
| | [shopify__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refunds) or<br>[shopify_gql__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refunds) | New model | | | New final model at the refund transaction grain. Aggregates line item financials and includes discrepancy adjustments for accurate reconciliation. | | ||
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | Data change | `refund_subtotal`, `refund_total_tax`, and `count_orders_with_refunds` bucketed by order creation date | Now bucketed by refund creation date | **Possible breaking change** New refund metrics now likely to change values. Aligns with Shopify Finance report behavior, which attributes refunds to the date the refund was issued. | | ||
| | [shopify__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customers)<br> and [shopify__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customer_emails) or<br>[shopify_gql__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customers)<br> and [shopify_gql__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customer_emails) | Data change | `lifetime_total_discount` and `avg_discount_per_order` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | |
There was a problem hiding this comment.
| | [shopify__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customers)<br> and [shopify__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customer_emails) or<br>[shopify_gql__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customers)<br> and [shopify_gql__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customer_emails) | Data change | `lifetime_total_discount` and `avg_discount_per_order` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | | |
| | [shopify__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customers)<br> and [shopify__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customer_emails) or<br>[shopify_gql__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customers)<br> and [shopify_gql__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customer_emails) | Data change | `lifetime_total_discount` and `avg_discount_per_order` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change:** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | |
| | [shopify__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__refunds) or<br>[shopify_gql__refunds](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__refunds) | New model | | | New final model at the refund transaction grain. Aggregates line item financials and includes discrepancy adjustments for accurate reconciliation. | | ||
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | Data change | `refund_subtotal`, `refund_total_tax`, and `count_orders_with_refunds` bucketed by order creation date | Now bucketed by refund creation date | **Possible breaking change** New refund metrics now likely to change values. Aligns with Shopify Finance report behavior, which attributes refunds to the date the refund was issued. | | ||
| | [shopify__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customers)<br> and [shopify__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customer_emails) or<br>[shopify_gql__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customers)<br> and [shopify_gql__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customer_emails) | Data change | `lifetime_total_discount` and `avg_discount_per_order` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | | ||
| | [shopify__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__products) or<br>[shopify_gql__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__products) | Data change | `product_total_discount` and `product_avg_discount_per_order_line` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | |
There was a problem hiding this comment.
| | [shopify__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__products) or<br>[shopify_gql__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__products) | Data change | `product_total_discount` and `product_avg_discount_per_order_line` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | | |
| | [shopify__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__products) or<br>[shopify_gql__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__products) | Data change | `product_total_discount` and `product_avg_discount_per_order_line` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change:** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | |
| | [shopify__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__products) or<br>[shopify_gql__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__products) | Data change | `product_total_discount` and `product_avg_discount_per_order_line` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | | ||
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | New fields | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics for the GQL path, matching the REST `shopify__daily_shop` columns. | | ||
| | [shopify__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__orders) or<br>[shopify_gql__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__orders) | New fields | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics at the order level, matching the REST `shopify__orders` columns. | | ||
| | [stg_shopify_gql__order_line_refund](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__order_line_refund) | New columns | | `price_pres_amount`, `price_pres_currency_code`, `price_shop_amount`, `price_shop_currency_code`, `is_restocked` | **Possible breaking change** Users with `order_line_refund_pass_through_columns` configured may encounter duplicate column errors if any passthrough columns overlap with the new columns. | |
There was a problem hiding this comment.
| | [stg_shopify_gql__order_line_refund](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__order_line_refund) | New columns | | `price_pres_amount`, `price_pres_currency_code`, `price_shop_amount`, `price_shop_currency_code`, `is_restocked` | **Possible breaking change** Users with `order_line_refund_pass_through_columns` configured may encounter duplicate column errors if any passthrough columns overlap with the new columns. | | |
| | [stg_shopify_gql__order_line_refund](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__order_line_refund) | New columns | | `price_pres_amount`, `price_pres_currency_code`, `price_shop_amount`, `price_shop_currency_code`, `is_restocked` | **Possible breaking change:** Users with `order_line_refund_pass_through_columns` configured may encounter duplicate column errors if any passthrough columns overlap with the new columns. | |
Also I'd move this up with the other Possible Breaking Changes
There was a problem hiding this comment.
Updated and moved.
| | [shopify__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__orders) or<br>[shopify_gql__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__orders) | New fields | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics at the order level, matching the REST `shopify__orders` columns. | | ||
| | [stg_shopify_gql__order_line_refund](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__order_line_refund) | New columns | | `price_pres_amount`, `price_pres_currency_code`, `price_shop_amount`, `price_shop_currency_code`, `is_restocked` | **Possible breaking change** Users with `order_line_refund_pass_through_columns` configured may encounter duplicate column errors if any passthrough columns overlap with the new columns. | | ||
| | [stg_shopify_gql__refund](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__refund) | New column | | `updated_at` | Timestamp of the last update to the refund record. | | ||
| | [stg_shopify_gql__return](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__return), [stg_shopify_gql__return_line_item](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__return_line_item), [stg_shopify_gql__return_shipping_fee](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__return_shipping_fee) | New models | | | Opt-in GraphQL staging models for return data. Can be enabled/disabled using `shopify_gql_using_return` variable. When enabled, `shopify_gql__refund_lines` is enriched with return lifecycle and return line item detail columns, and `shopify_gql__refunds` is enriched with return context and return shipping fee columns. | |
There was a problem hiding this comment.
Would be good to include links to the README section for these variables
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | New fields | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics for the GQL path, matching the REST `shopify__daily_shop` columns. | | ||
| | [shopify__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__orders) or<br>[shopify_gql__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__orders) | New fields | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics at the order level, matching the REST `shopify__orders` columns. | |
There was a problem hiding this comment.
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | New fields | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics for the GQL path, matching the REST `shopify__daily_shop` columns. | | |
| | [shopify__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__orders) or<br>[shopify_gql__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__orders) | New fields | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics at the order level, matching the REST `shopify__orders` columns. | | |
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | New columns | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics for the GQL path, matching the REST `shopify__daily_shop` columns. | | |
| | [shopify__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__orders) or<br>[shopify_gql__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__orders) | New columns | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics at the order level, matching the REST `shopify__orders` columns. | |
To standardize with the following rows
There was a problem hiding this comment.
Standardized.
fivetran-avinash
left a comment
There was a problem hiding this comment.
@fivetran-jamie ready for re-review once Buildkite passes
fivetran-jamie
left a comment
There was a problem hiding this comment.
Left a few comments! Approving as they shouldn't block this
| | [shopify__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customers)<br> and [shopify__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__customer_emails) or<br>[shopify_gql__customers](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customers)<br> and [shopify_gql__customer_emails](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__customer_emails) | Data change | `lifetime_total_discount` and `avg_discount_per_order` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change:** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | | ||
| | [shopify__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__products) or<br>[shopify_gql__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__products) | Data change | `product_total_discount` and `product_avg_discount_per_order_line` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change:** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | | ||
| | [stg_shopify_gql__order_line_refund](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__order_line_refund) | New columns | | `price_pres_amount`, `price_pres_currency_code`, `price_shop_amount`, `price_shop_currency_code`, `is_restocked` | **Possible breaking change:** Users with `order_line_refund_pass_through_columns` configured may encounter duplicate column errors if any passthrough columns overlap with the new columns. | | ||
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | New columns | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics for the GQL path, matching the REST `shopify__daily_shop` columns. | |
There was a problem hiding this comment.
The note is a lil confusing since it says we're adding the fields to match shopify__daily_shop, but shopify__daily_shop itself is included here and had new columns added.
I think it would be fine to just say "New finance metrics" or something
| | [shopify__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__products) or<br>[shopify_gql__products](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__products) | Data change | `product_total_discount` and `product_avg_discount_per_order_line` sourced from `order_line.total_discount` | Now sourced from `discount_allocation` | **Possible breaking change:** New discount metrics now likely to change values. `discount_allocation` is the more reliable source for order line discount values. | | ||
| | [stg_shopify_gql__order_line_refund](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.stg_shopify_gql__order_line_refund) | New columns | | `price_pres_amount`, `price_pres_currency_code`, `price_shop_amount`, `price_shop_currency_code`, `is_restocked` | **Possible breaking change:** Users with `order_line_refund_pass_through_columns` configured may encounter duplicate column errors if any passthrough columns overlap with the new columns. | | ||
| | [shopify__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__daily_shop) or<br>[shopify_gql__daily_shop](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__daily_shop) | New columns | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics for the GQL path, matching the REST `shopify__daily_shop` columns. | | ||
| | [shopify__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify__orders) or<br>[shopify_gql__orders](https://fivetran.github.io/dbt_shopify/#!/model/model.shopify.shopify_gql__orders) | New columns | | `gross_sales`, `discounts`, `returns`, `net_sales` | Finance metrics at the order level, matching the REST `shopify__orders` columns. | |
There was a problem hiding this comment.
Similar confusion about the note -- shopify__orders is included here as having had fields added, but it says we're adding the fields to match shopify__orders
| sum(coalesce(discount_allocation_agg.discount_allocation_amount, 0)) as product_total_discount, | ||
| sum(order_lines.order_line_tax) as product_total_tax, | ||
| avg(order_lines.quantity) as avg_quantity_per_order_line, | ||
| avg(order_lines.total_discount) as product_avg_discount_per_order_line, | ||
| avg(coalesce(discount_allocation_agg.discount_allocation_amount, 0)) as product_avg_discount_per_order_line, |
There was a problem hiding this comment.
Ah I mean in the CHANGELOG, as some values may be changing from null to 0. Could go in the Under the Hood section
Not of the utmost importance but could be good to have there
PR Overview
Package version introduced in this PR:
This PR addresses the following Issue/Feature(s):
Summary of changes:
Submission Checklist
downstream_model_column_countmacro?Changelog