Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: validate kustomize build of tasks and pipelines #1792

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tisutisu
Copy link
Contributor

@tisutisu tisutisu commented Jan 7, 2025

This PR adds a CI check to validate kustomize build is successful and also checks that PR authors are committing built manifests files of task/pipeline change as well with their task/pipeline changes.
Part of story: https://issues.redhat.com/browse/STONEBLD-3042

@tisutisu tisutisu marked this pull request as ready for review January 7, 2025 08:11
@tisutisu tisutisu requested a review from a team as a code owner January 7, 2025 08:11
@tisutisu tisutisu force-pushed the kustomize-build branch 3 times, most recently from 29e718a to fe32af3 Compare January 7, 2025 12:45
cd "$SCRIPT_DIR/.."
task_dirs=$(find task -maxdepth 4 \( -name 'kustomization.yaml' -o -name 'kustomization.yml' \) -exec dirname {} \;)
pipeline_dirs=$(find pipelines -maxdepth 3 \( -name 'kustomization.yaml' -o -name 'kustomization.yml' \) -exec dirname {} \;)
local out=auto-generated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of prepending the auto-generated/ prefix, can we put the generated files into the original directory? e.g. task/build-image-index/0.1/build-image-index.yaml?

I think that would automatically make the build-and-push.sh script notice that the task yaml already exists and it's not necessary to run kustomize again

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, we have to also update the build-and-push.sh script and don't run the oc kustomize cmds again during build-and-push.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we don't need to modify the build-and-push script, if we commit the tasks and pipelines to the same directory, made the changes now.

@tisutisu tisutisu marked this pull request as draft January 9, 2025 12:32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This task is mirrored from another repo, so changes made directly will eventually be overwritten by changes made there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will ignore building manifest for this task then

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This task is mirrored from another repo, so changes made directly will eventually be overwritten by changes made there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for this task as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This task is mirrored from another repo, so changes made directly will eventually be overwritten by changes made there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for this task as well

@tisutisu tisutisu force-pushed the kustomize-build branch 2 times, most recently from 55f6f38 to 04430c6 Compare January 13, 2025 03:51
@tisutisu tisutisu marked this pull request as ready for review January 13, 2025 03:53
Copy link
Contributor

@tkdchen tkdchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please describe the solution in the commit message?


cd "$SCRIPT_DIR/.."
task_dirs=$(find task -maxdepth 4 \( -name 'kustomization.yaml' -o -name 'kustomization.yml' \) -exec dirname {} \;)
pipeline_dirs=$(find pipelines -mindepth 2 \( -name 'kustomization.yaml' -o -name 'kustomization.yml' \) -exec dirname {} \;)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The task directory structure is fixed. Can you keep consistency with what build-and-push.sh does to access the tasks and pipelines directories?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the build-manifests.sh script, we are trying to find the task which has kustomization.yaml inside it. while in the build-and-push.sh we are building bundle and pushing for all the tasks, there is some difference. Do you see any problem in the query or want me to write the query to find the tasks in different way?

echo "Please run ./hack/build-manifests.sh and update your PR" >&2;
exit 1;
}
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like ./hack/build-manifests.sh becomes a mandatory part of the development workflow when working with tasks and pipelines. Can you describe this? Probably inside the build-manifests.sh script itself at least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Described it in the README change here. Do you think it is sufficient or need to be added to the build-manifests.sh script as well?

@tisutisu
Copy link
Contributor Author

/retest

@tisutisu
Copy link
Contributor Author

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants