feature/remove-age (#28)

* feature/remove-age * update changelog, decisionlog, readme * regen docs * update changelog * Apply suggestions from code review Co-authored-by: Joe Markiewicz <[email protected]> * Apply suggestions from code review Co-authored-by: Jamie Rodriguez <[email protected]> * update changelog --------- Co-authored-by: Joe Markiewicz <[email protected]> Co-authored-by: Jamie Rodriguez <[email protected]>
fivetran · Dec 9, 2024 · 389c549 · 389c549
1 parent 4a7a201
commit 389c549
Show file tree

Hide file tree

Showing 11 changed files with 44 additions and 50 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,14 @@
+# dbt_tiktok_ads_source v0.7.0
+[PR #28](https://github.com/fivetran/dbt_tiktok_ads_source/pull/28) includes the following updates:
+
+## Breaking Changes
+- In the [July 2023 TikTok Ads connector update](https://fivetran.com/docs/connectors/applications/tiktok-ads/changelog#july2023) for the `ADGROUP_HISTORY` table, the `age` column was renamed to `age_groups`.
+  - Previously, we coalesced these two columns in the `stg_tiktok_ads__ad_group_history` model to account for connectors using the old naming convention. However, due to inconsistent data types (string and JSON), we can no longer use this approach.
+  - As a result, the coalesced field has been removed in favor of the `age_groups` column.
+  - If necessary, you can populate historical data in the `age_groups` column by performing a resync of the `ADGROUP_HISTORY` table, since TikTok provides all data regardless of the previous sync state.
+  - For more details, see the [DECISIONLOG entry](https://github.com/fivetran/dbt_tiktok_ads_source/blob/main/DECISIONLOG.md#age_groups-and-age-columns).
+
+
 # dbt_tiktok_ads_source v0.6.0
 [PR #23](https://github.com/fivetran/dbt_tiktok_ads_source/pull/23) includes the following **BREAKING CHANGE** updates:
 

diff --git a/DECISIONLOG.md b/DECISIONLOG.md
@@ -1,4 +1,20 @@
+# DECISION LOG
+
 ## Why don't metrics add up across different grains (Ex. ad level vs campaign level)?
 Not all ads are served at the ad level. In other words, there are some ads that are served only at the ad group, campaign, etc. levels. The implications are that since not ads are included in the ad-level report, their associated spend, for example, won't be included at that grain. Therefore your spend totals may differ across the ad grain and another grain. 
 
-This is a reason why we have broken out the ad reporting packages into separate hierarchical end models (Ad, Ad Group, Campaign, and more). Because if we only used ad-level reports, we could be missing data.s
+This is a reason why we have broken out the ad reporting packages into separate hierarchical end models (Ad, Ad Group, Campaign, and more). Because if we only used ad-level reports, we could be missing data.
+
+## `age_groups` and `age` columns
+In the [July 2023 TikTok Ads connector update](https://fivetran.com/docs/connectors/applications/tiktok-ads/changelog#july2023), we renamed the `age` column in the `ADGROUP_HISTORY` table to `age_groups`. To maintain compatibility with connectors using the old column name, our data models previously coalesced the `age` and `age_groups` columns. However, due to inconsistent data types between `age` and `age_groups`, this approach began causing errors.
+
+### Decision
+- The `age` column has been removed from the `stg_tiktok_ads__ad_group_history` model and the `fill` staging macro for the ad group history table.
+- Models now rely exclusively on the `age_groups` column.
+
+### Impact
+- Customers using the `age` column will no longer have access to this field in the models.
+- The change simplifies logic and resolves data inconsistency issues caused by merging fields with different data types.
+
+### Action for Customers Requiring Historical `age` Data
+Customers who still need the historical `age` column data can resync the `ADGROUP_HISTORY` table in their TikTok Ads connector. TikTok provides all historical data in the `age_groups` column, allowing the data to be fully populated.
diff --git a/README.md b/README.md
@@ -41,7 +41,7 @@ If you  are **not** using the [Tiktok transformation package](https://github.com
 ```yaml
 packages:
   - package: fivetran/tiktok_ads_source
-    version: [">=0.6.0", "<0.7.0"]
+    version: [">=0.7.0", "<0.8.0"]
 ```
 
 ### Step 3: Define database and schema variables

diff --git a/dbt_project.yml b/dbt_project.yml
@@ -1,5 +1,5 @@
 name: 'tiktok_ads_source'
-version: '0.6.0'
+version: '0.7.0'
 config-version: 2
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 models:

diff --git a/docs/catalog.json b/docs/catalog.json
diff --git a/docs/index.html b/docs/index.html
diff --git a/docs/manifest.json b/docs/manifest.json
diff --git a/integration_tests/dbt_project.yml b/integration_tests/dbt_project.yml
@@ -1,5 +1,5 @@
 name: 'tiktok_ads_source_integration_tests'
-version: '0.6.0'
+version: '0.7.0'
 profile: 'integration_tests'
 config-version: 2
 

diff --git a/macros/get_ad_group_history_columns.sql b/macros/get_ad_group_history_columns.sql
@@ -17,7 +17,6 @@
     {"name": "updated_at", "datatype": dbt.type_timestamp()},
     {"name": "interest_category_v_2", "datatype": dbt.type_string()},
     {"name": "action_categories", "datatype": dbt.type_string()},
-    {"name": "age", "datatype": dbt.type_string()},
     {"name": "age_groups", "datatype": dbt.type_string()},
     {"name": "languages", "datatype": dbt.type_string()}
 

diff --git a/models/src_tiktok_ads.yml b/models/src_tiktok_ads.yml
@@ -130,11 +130,6 @@ sources:
             description: The number of days of the time period to include action from.
           - name: adgroup_name
             description: Ad group name. Character limit is 512 and cannot contain emoji.
-          - name: age
-            description: >
-              Age groups you want to target. 
-              IMPORTANT: This field will not be populated for connectors utilizing the Tiktok Ads v1.3 API version, as the column was renamed. 
-              The new column name is `age_group`.
           - name: age_groups
             description: >
               Age groups you want to target.

diff --git a/models/stg_tiktok_ads__ad_group_history.sql b/models/stg_tiktok_ads__ad_group_history.sql
@@ -36,7 +36,7 @@ final as (
         action_days,
         action_categories,
         adgroup_name as ad_group_name,
-        coalesce(age_groups, age) as age_groups,
+        age_groups,
         audience_type,
         budget,
         category,