Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Investigate issue in the intersection of "+tag:some_tag" select statements #1466

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

tatiana
Copy link
Collaborator

@tatiana tatiana commented Jan 14, 2025

A user reported that they see the correct DbtDag when using Cosmos 1.8.1 with:

  • LoadMode.DBT_LS
  • RenderConfig(selector="accounts_marts")

Where the selector accounts_marts is defined as:

    - name: accounts_marts
      description: Run Accounts models
      definition:
        intersection:
          - '+tag:accounts'
          - '+tag:datamart'
          - '+tag:stratus'

The expected behaviour includes:

  • 164 Airflow tasks
  • 152 Local run tasks
  • 12 Snapshot tasks

However, when they attempt to run the same DbtDag using:

  • LoadMode.DBT_MANIFEST
  • RenderConfig(select=["+tag:accounts,+tag:datamart,+tag:stratus"])

Their DbtDag seems to have the wrong subset of nodes.

They reported:

  • 197 Airflow tasks
  • 183 Local run tasks
  • 14 Snapshot tasks

This pull request aims to reproduce and fix this issue.

Copy link

netlify bot commented Jan 14, 2025

Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name Link
🔨 Latest commit 4b693a6
🔍 Latest deploy log https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/6786676b45a18500084febc4

Copy link

cloudflare-workers-and-pages bot commented Jan 14, 2025

Deploying astronomer-cosmos with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4b693a6
Status: ✅  Deploy successful!
Preview URL: https://847acfc7.astronomer-cosmos.pages.dev
Branch Preview URL: https://select-intersection-graphs.astronomer-cosmos.pages.dev

View logs

@tatiana tatiana changed the title WIP: Add support for intersection of "+tag:some_tag" in RenderConfig(select) WIP: Investigate issue in the intersection of "+tag:some_tag" in RenderConfig(select) Jan 14, 2025
@tatiana tatiana changed the title WIP: Investigate issue in the intersection of "+tag:some_tag" in RenderConfig(select) WIP: Investigate issue in the intersection of "+tag:some_tag" select statements Jan 14, 2025
Copy link

codecov bot commented Jan 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.94%. Comparing base (768a0b9) to head (4b693a6).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1466   +/-   ##
=======================================
  Coverage   96.94%   96.94%           
=======================================
  Files          73       73           
  Lines        4355     4355           
=======================================
  Hits         4222     4222           
  Misses        133      133           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant