Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update variable declarations #155

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

fivetran-avinash
Copy link
Contributor

@fivetran-avinash fivetran-avinash commented Jan 3, 2025

PR Overview

This PR will address the following Issue/Feature: Internal height ticket

This PR will result in the following new package version: 0.17.1

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Macro Updates

  • Introduced a local version of the persist_pass_through_columns macro that directly calls the variables within our models. This removes the existing string-to-variable conversion and leads to cleaner parsing.
  • This new macro is applied to all end models with passthrough column functionality, and replaces the existing persist_pass_through_columns macro.
  • Models impacted for both netsuite__* and netsuite2__* include balance_sheet, income_statement, transaction_details.
  • The process for adding passthrough columns remains unchanged. Consult the README for more details.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

Ran on multiple variations of passthrough columns (with name, with alias, with multiple passthrough columns, with no passthrough columns).

If you had to summarize this PR in an emoji, which would it be?

🚲

@fivetran-avinash fivetran-avinash self-assigned this Jan 6, 2025
@fivetran-avinash fivetran-avinash marked this pull request as ready for review January 6, 2025 21:29
Copy link
Contributor

@fivetran-sheringuyen fivetran-sheringuyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the scope of the pass through columns issue - it looks like it's now working in our environment! I'm not sure I'm the right person to approve a release against this repo though.

@@ -11,9 +11,9 @@ vars:
netsuite_schema: netsuite_integration_tests_8
netsuite_data_model_override: netsuite

# # Enable below when generating docs
# netsuite2__multibook_accounting_enabled: true
# netsuite2__using_to_subsidiary: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obligatory reminder: Does this need to be uncommented before merge?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this only needs to enabled when doing the docs generation. After that, it should be commented back out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh i meant re-commented*

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh d'oh. Yes, should be recommented.

@@ -1,3 +1,4 @@
packages:
- package: fivetran/netsuite_source
version: [">=0.11.0", "<0.12.0"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obligatory reminder to update this before release

Comment on lines +2 to +4
revision: update/netsuite-variable-declarations
warn-unpinned: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- git: https://github.com/fivetran/dbt_netsuite_source.git
revision: update/netsuite-variable-declarations
warn-unpinned: false
- package: fivetran/netsuite_source
version: [">=0.11.0", "<0.12.0"]

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @fivetran-avinash, a few questions and comments before approval.

[PR #155](https://github.com/fivetran/dbt_netsuite/pull/155) includes the following updates:

## Macro Updates
- Introduced a local version of the `persist_pass_through_columns` macro that directly calls the variables within our models. This removes the existing string-to-variable conversion and leads to cleaner parsing.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the comment from the source, can we include a note here that no changes are required by the customer following this update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

@@ -67,19 +67,19 @@ income_statement as (
subsidiaries.name as subsidiary_name

--The below script allows for accounts table pass through columns.
{{ fivetran_utils.persist_pass_through_columns('accounts_pass_through_columns', identifier='accounts') }},
{{ netsuite.persist_pass_through_columns(var('accounts_pass_through_columns', none), identifier='accounts') }},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we use None here and not [] like we usually do?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Especially since that's how we have it defined in the dbt_project.yml

Copy link
Contributor Author

@fivetran-avinash fivetran-avinash Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I was following what was defined in our persist passthrough macro code and assumed the none condition was what we wanted. Updated this in all macro calls.

Copy link
Contributor Author

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-joemarkiewicz This is ready for re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants