Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: map artifacts to commits via repo tags #508

Merged
merged 44 commits into from
Dec 18, 2023

Conversation

benmss
Copy link
Member

@benmss benmss commented Oct 10, 2023

This feature allows artifacts with a specified version to be mapped to the repository commit they were created from.
The mapping is achieved through use of repository tags, where these tags match the specified version sufficiently.
Tag to version comparisons are performed using regular expression.

Tags are expected to be of the format (with allowances for extra characters):
[<artifact_name>-]<version>

artifact_name is an optional prefix that matches the name of the related artifact, e.g. "commons-io".

<version> is a string of some number of alphanumeric characters that can be (multi-) separated by any number of non-numeric characters.

Examples of accepted tag versions:

  • v1.2.3
  • 1_2_3
  • 1.2.3-DEV
  • 1.2.3.RELEASE
  • r1rv23
  • 1.2.3.v123123

Further changes to this PR are in progress:

  • Support tags whose suffix does not match their related version
  • Do not fallback to latest commit if the commit finder fails
  • Use default branch for commits with no associated branch (for now)
  • Use purls with versions instead of commits in integration tests
  • Allow for purls that represent repositories not just artefacts
  • Extend test file to include purls and comments
  • Use Hypothesis to test commit finder input

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 10, 2023
@benmss

This comment was marked as outdated.

src/macaron/repo_finder/repo_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/repo_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/repo_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/repo_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/repo_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/repo_finder.py Outdated Show resolved Hide resolved
tests/repo_finder/test_repo_finder.py Outdated Show resolved Hide resolved
tests/repo_finder/test_repo_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/commit_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/commit_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/commit_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/commit_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/commit_finder.py Outdated Show resolved Hide resolved
src/macaron/repo_finder/commit_finder.py Show resolved Hide resolved
tromai
tromai previously approved these changes Oct 19, 2023
Copy link
Member

@behnazh-w behnazh-w left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like for pkg:maven/com.google.guava/[email protected]?type=jar the commit finder maps the artifact to (adde55ee1f9bc7fb053183b5465f413547c198fe) which is the latest commit on master.

Can you please create tasks for the following items in the PR description?

  1. We should cover cases where artifact versions have suffixes that do not match tags.
  2. If a tag is not found, as we had discussed before, we shouldn't fall back to the latest commit. Instead, the repository should be None to fit it the current data model.
  3. Since we have a commit mapping feature, we shouldn't need to pin tests in the integration tests to a commit anymore. Just passing the purl should be enough. So, we can probably remove at least some of the -c config.yaml runs in the integration tests with purl

Here is one dependency from apache_maven_root_sbom.json which we use in our integration tests that is also mapped to the latest commit: pkg:maven/org.junit.jupiter/[email protected]?type=jar

…on of string; simplify last array item lookup and string prefix removal; fix failed string replacement

Signed-off-by: Ben Selwyn-Smith <[email protected]>
…e commit finder tests to new file

Signed-off-by: Ben Selwyn-Smith <[email protected]>
…t of loop and extended to safely support three extensions

Signed-off-by: Ben Selwyn-Smith <[email protected]>
…tains a version-like part

Signed-off-by: Ben Selwyn-Smith <[email protected]>
…commit finder unit testing

Signed-off-by: Ben Selwyn-Smith <[email protected]>
…tern; allow exact artefact name as a valid prefix

Signed-off-by: Ben Selwyn-Smith <[email protected]>
…ved surplus check from commit finder test

Signed-off-by: Ben Selwyn-Smith <[email protected]>
…ge; update e2e test file path; extract purl type check into standalone function and add unit test

Signed-off-by: Ben Selwyn-Smith <[email protected]>
@benmss benmss force-pushed the 430-artifact-to-commit-mapping branch from bf9bcb7 to c27bd35 Compare November 29, 2023 17:00
Signed-off-by: Ben Selwyn-Smith <[email protected]>
Copy link
Member

@nathanwn nathanwn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's an LGTM for now. Thanks for your work.

@benmss benmss merged commit 1f7f664 into staging Dec 18, 2023
9 checks passed
art1f1c3R pushed a commit that referenced this pull request Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants