Skip to content

Commit

Permalink
update/default-schema-change (#12)
Browse files Browse the repository at this point in the history
* update/default-schema-change

* changelog update

* additional variable breaking changes

* seed rename

* docs regen

* identifier in readme fix

* Update integration_tests/dbt_project.yml

Co-authored-by: Avinash Kunnath <[email protected]>

* Update README.md

---------

Co-authored-by: Avinash Kunnath <[email protected]>
  • Loading branch information
fivetran-joemarkiewicz and fivetran-avinash authored Aug 27, 2024
1 parent 228089a commit 17c4607
Show file tree
Hide file tree
Showing 24 changed files with 104 additions and 112 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,6 @@ body:
description: Our team will assess this issue and let you know if we will add it to a future sprint. However, if you would like to expedite the solution, we encourage you to contribute to the package via a PR. Our team will then work with you to approve and merge your contributions as soon as possible.
options:
- label: Yes.
- label: Yes, but I will need assistance and will schedule time during our [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance
- label: Yes, but I will need assistance.
- label: No.
required: false
3 changes: 0 additions & 3 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
contact_links:
- name: Ask a question during our office hours
url: https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours
about: Schedule time during the external office hours block with the Fivetran Analytics Engineering team for support
- name: Fivetran connector question
url: https://support.fivetran.com/hc
about: Have a question about your connector? Check out the Fivetran support portal for more details.
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ body:
description: Our team will assess this feature and let you know if we will add it to a future sprint. However, if you would like to expedite the feature, we encourage you to contribute to the package via a PR. Our team will then work with you to approve and merge your contributions as soon as possible.
options:
- label: Yes.
- label: Yes, but I will need assistance and will schedule time during your [office hours](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) for guidance.
- label: Yes, but I will need assistance.
- label: No.
required: false
- type: textarea
Expand Down
43 changes: 11 additions & 32 deletions .github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,48 +4,27 @@
**This PR will result in the following new package version:**
<!--- Please add details around your decision for breaking vs non-breaking version upgrade. If this is a breaking change, were backwards-compatible options explored? -->

**Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:**
**Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:**
<!--- Copy/paste the CHANGELOG for this version below. -->

## PR Checklist
### Basic Validation
Please acknowledge that you have successfully performed the following commands locally:
- [ ] dbt compile
- [ ] dbt run –full-refresh
- [ ] dbt run
- [ ] dbt test
- [ ] dbt run –vars (if applicable)
- [ ] dbt run –full-refresh && dbt test
- [ ] dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:
- [ ] The appropriate issue has been linked and tagged
- [ ] You are assigned to the corresponding issue and this PR
- [ ] The appropriate issue has been linked, tagged, and properly assigned
- [ ] All necessary documentation and version upgrades have been applied
<!--- Be sure to update the package version in the dbt_project.yml, integration_tests/dbt_project.yml, and README if necessary. -->
- [ ] docs were regenerated (unless this PR does not include any code or yml updates)
- [ ] BuildKite integration tests are passing
- [ ] Detailed validation steps have been provided below

### Detailed Validation
Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":
- [ ] You have validated these changes and assure this PR will address the respective Issue/Feature.
- [ ] You are reasonably confident these changes will not impact any other components of this package or any dependent packages.
- [ ] You have provided details below around the validation steps performed to gain confidence in these changes.
Please share any and all of your validation steps:
<!--- Provide the steps you took to validate your changes below. -->

### Standard Updates
Please acknowledge that your PR contains the following standard updates:
- Package versioning has been appropriately indexed in the following locations:
- [ ] indexed within dbt_project.yml
- [ ] indexed within integration_tests/dbt_project.yml
- [ ] CHANGELOG has individual entries for each respective change in this PR
<!--- If there is a parallel upstream change, remember to reference the corresponding CHANGELOG as an individual entry. -->
- [ ] README updates have been applied (if applicable)
<!--- Remember to check the following README locations for common updates. →
<!--- Suggested install range (needed for breaking changes) →
<!--- Dependency matrix is appropriately updated (if applicable) →
<!--- New variable documentation (if applicable) -->
- [ ] DECISIONLOG updates have been updated (if applicable)
- [ ] Appropriate yml documentation has been added (if applicable)

### dbt Docs
Please acknowledge that after the above were all completed the below were applied to your branch:
- [ ] docs were regenerated (unless this PR does not include any code or yml updates)

### If you had to summarize this PR in an emoji, which would it be?
<!--- For a complete list of markdown compatible emojis check our this git repo (https://gist.github.com/rxaviers/7360908) -->
:dancer:
:dancer:
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,4 @@ We are so excited you decided to contribute to the Fivetran community dbt packag
**PR Template**
- [Community Pull Request Template](?expand=1&template=pull_request_template.md) (default)

- [Maintainer Pull Request Template](?expand=1&template=maintainer_pull_request_template.md) (to be used by maintainers)
- [Maintainer Pull Request Template](?expand=1&template=maintainer_pull_request_template.md) (to be used by maintainers)
13 changes: 13 additions & 0 deletions .github/workflows/auto-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: 'auto release'
on:
pull_request:
types:
- closed
branches:
- main

jobs:
call-workflow-passing-data:
if: github.event.pull_request.merged
uses: fivetran/dbt_package_automations/.github/workflows/auto-release.yml@main
secrets: inherit
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@ target/
dbt_modules/
logs/
.DS_Store
dbt_packages/
dbt_packages/
env/
package-lock.yml
32 changes: 30 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,36 @@
# dbt_twitter_organic_source v0.UPDATE.UPDATE
# dbt_twitter_organic_source v0.3.0

## Breaking Change
[PR #12](https://github.com/fivetran/dbt_twitter_organic_source/pull/12) includes the following breaking change updates:

- The source defined in the `src_twitter_organic.yml` file has been renamed from `twitter_organic` to `twitter` to align with the default schema name used by the upstream Fivetran connector.
- If you're referencing sources from this package, please update your source references as needed. See below for the full scope of source changes.

| **New Source Reference** | **Old Source Reference** |
|----------------------------------|----------------------------------|
| `"{{ source('twitter','account_history') }}"` | `"{{ source('twitter_organic','account_history') }}"` |
| `"{{ source('twitter','organic_tweet_report') }}"` | `"{{ source('twitter_organic','organic_tweet_report') }}"` |
| `"{{ source('twitter','tweet') }}"` | `"{{ source('twitter_organic','tweet') }}"` |
| `"{{ source('twitter','twitter_user_history') }}"` | `"{{ source('twitter_organic','twitter_user_history') }}"` |

- The default schema name has been modified from `twitter_organic` to now be `twitter` to more closely align with the default schema name generated by the Fivetran connector. Please be aware if you were leveraging the previous default schema then you will need to update the `twitter_organic_schema` variable accordingly.
- All identifier variables in the `src_twitter_organic.yml` file have been renamed. If you’re using any of these in your project, please update them accordingly. The changes include:
- Prepending `twitter_organic_*` has been updated to `twitter_*` to align with the schema change.
- The spelling of `*_identifer` has been corrected to `*_identifier`.

| **New Identifier Variable Name** | **Old Identifier Variable Name** |
|----------------------------------|----------------------------------|
| `twitter_account_history_identifier` | `twitter_organic_account_history_identifer` |
| `twitter_organic_tweet_report_identifier` | `twitter_organic_organic_tweet_report_identifer` |
| `twitter_tweet_identifier` | `twitter_organic_tweet_identifer` |
| `twitter_twitter_user_history_identifier` | `twitter_organic_twitter_user_history_identifer` |

## Under the Hood:
- All tmp models have been updated to reference the new source name `twitter` (previously `twitter_organic`). ([#12](https://github.com/fivetran/dbt_twitter_organic_source/pull/12))
- Renamed the seed files to allow for more testing functionality. ([#12](https://github.com/fivetran/dbt_twitter_organic_source/pull/12))
- Included auto-releaser GitHub Actions workflow to automate future releases. ([#12](https://github.com/fivetran/dbt_twitter_organic_source/pull/12))
- Updated the maintainer PR, Issue, Feature Request, and Config templates to resemble the most up to date format. ([#12](https://github.com/fivetran/dbt_twitter_organic_source/pull/12))
- Incorporated the new `fivetran_utils.drop_schemas_automation` macro into the end of each Buildkite integration test job. ([#9](https://github.com/fivetran/dbt_twitter_organic_source/pull/9))
- Updated the pull request [templates](/.github). ([#9](https://github.com/fivetran/dbt_twitter_organic_source/pull/9))

# dbt_twitter_organic_source v0.2.0

Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,11 @@ Include the following twitter_organic_source package version in your `packages.y
```yaml
packages:
- package: fivetran/twitter_organic_source
version: [">=0.2.0", "<0.3.0"] # we recommend using ranges to capture non-breaking changes automatically
version: [">=0.3.0", "<0.4.0"] # we recommend using ranges to capture non-breaking changes automatically
```

## Step 3: Define database and schema variables
By default, this package will look for your Twitter Organic data in the `twitter_organic` schema of your [target database](https://docs.getdbt.com/docs/running-a-dbt-project/using-the-command-line-interface/configure-your-profile). If this is not where your Twitter Organic data is, please add the following configuration to your `dbt_project.yml` file:
By default, this package will look for your Twitter Organic data in the `twitter` schema of your [target database](https://docs.getdbt.com/docs/running-a-dbt-project/using-the-command-line-interface/configure-your-profile). If this is not where your Twitter Organic data is, please add the following configuration to your `dbt_project.yml` file:

```yml
vars:
Expand All @@ -54,7 +54,7 @@ vars:
```

## (Optional) Step 4: Additional Configurations
<details><summary>Expand for configurations</summary>
<details open><summary>Expand for configurations</summary>

### Changing the Build Schema

Expand All @@ -72,7 +72,7 @@ If an individual source table has a different name than the package expects, add

```yml
vars:
twitter_organic_<default_source_table_name>_identifier: your_table_name
twitter_<default_source_table_name>_identifier: your_table_name
```

### Unioning Multiple Twitter Organic Connectors
Expand Down Expand Up @@ -125,4 +125,4 @@ We highly encourage and welcome contributions to this package. Check out [this p
# 🏪 Are there any resources available?
- If you encounter any questions or want to reach out for help, please refer to the [GitHub Issue](https://github.com/fivetran/dbt_twitter_organic_source/issues/new/choose) section to find the right avenue of support for you.
- If you would like to provide feedback to the dbt package team at Fivetran, or would like to request a future dbt package to be developed, then feel free to fill out our [Feedback Form](https://www.surveymonkey.com/r/DQ7K7WW).
- Have questions or want to just say hi? Book a time during our office hours [here](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) or send us an email at [email protected].
- Have questions or want to just say hi? Book a time during our office hours [here](https://calendly.com/fivetran-solutions-team/fivetran-solutions-team-office-hours) or send us an email at [email protected].
10 changes: 5 additions & 5 deletions dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
name: 'twitter_organic_source'
version: '0.2.0'
version: '0.3.0'
config-version: 2
require-dbt-version: [">=1.3.0", "<2.0.0"]
vars:
account_history: "{{ source('twitter_organic','account_history') }}"
organic_tweet_report: "{{ source('twitter_organic','organic_tweet_report') }}"
tweet: "{{ source('twitter_organic','tweet') }}"
twitter_user_history: "{{ source('twitter_organic','twitter_user_history') }}"
account_history: "{{ source('twitter','account_history') }}"
organic_tweet_report: "{{ source('twitter','organic_tweet_report') }}"
tweet: "{{ source('twitter','tweet') }}"
twitter_user_history: "{{ source('twitter','twitter_user_history') }}"
models:
twitter_organic_source:
+schema: stg_twitter_organic
Expand Down
2 changes: 1 addition & 1 deletion docs/catalog.json

Large diffs are not rendered by default.

Loading

0 comments on commit 17c4607

Please sign in to comment.