feat: map artifacts to commits via repo tags (#508)

Signed-off-by: Ben Selwyn-Smith <[email protected]>

docs/source/pages/developers_guide/apidoc/macaron.repo_finder.rst

@@ -9,6 +9,14 @@ macaron.repo\_finder package
 Submodules
 ----------
 
+macaron.repo\_finder.commit\_finder module
+------------------------------------------
+
+.. automodule:: macaron.repo_finder.commit_finder
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 macaron.repo\_finder.repo\_finder module
 ----------------------------------------
 

docs/source/pages/using.rst

@@ -166,6 +166,10 @@ For more detailed information on converting a given artifact into a PURL, see `P
 
 .. note:: If a repository is not also provided, Macaron will try to discover it based on the artifact purl. For this to work, ``find_repos`` in the configuration file **must be enabled**\. See `Analyzing more dependencies <#more-deps>`_ for more information about the configuration options of the Repository Finding feature.
 
+.. note:: If no repository is provided, but the PURL contains a version (as with all of the above examples), Macaron will attempt to find the exact commit that matches the provided version. For this to work, the discovered repository must support and make use of tags to denote commits relating to released artifacts.
+
+
+
 -------------------------------------------------
 Verifying provenance expectations in CUE language
 -------------------------------------------------
@@ -247,9 +251,7 @@ With ``path_to_sbom`` is the path to the SBOM you want to use.
 Analyzing more dependencies
 '''''''''''''''''''''''''''
 
-In some cases the dependencies that Macaron discovers lack a direct connection to a repository for it to analyze. To improve results in these instances, the Repository Finding feature can be enabled. This feature makes use of a dependency's identifying information that can be found using the package registries located on the Internet.
-
-.. note:: The Repository Finding feature currently only works for Java projects via SCM meta data found within artifact POM files.
+In some cases the dependencies that Macaron discovers lack a direct connection to a repository for it to analyze. To improve results in these instances, Macaron is capable of automatically determining the repository and exact commit that matches the given dependency. For repositories, this is achieved through examination of SCM meta data found within artifact POM files (for Java), or use of Google's Open Source Insights API (for other languages). For commits, Macaron will attempt to match repository tags with the artifact version being sought, thereby requiring that the repository supports and uses tags on commits that were used for releases.
 
 This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution.
 

scripts/dev_scripts/integration_tests.sh

@@ -10,6 +10,7 @@ RESOURCES=$WORKSPACE/src/macaron/resources
 COMPARE_DEPS=$WORKSPACE/tests/dependency_analyzer/compare_dependencies.py
 COMPARE_JSON_OUT=$WORKSPACE/tests/e2e/compare_e2e_result.py
 TEST_REPO_FINDER=$WORKSPACE/tests/e2e/repo_finder/repo_finder.py
+TEST_COMMIT_FINDER=$WORKSPACE/tests/e2e/repo_finder/commit_finder.py
 RUN_MACARON="python -m macaron -o $WORKSPACE/output"
 RESULT_CODE=0
 UPDATE=0
@@ -277,7 +278,7 @@ declare -a COMPARE_FILES=(
     "slf4j.json"
 )
 
-$RUN_MACARON analyze -c $WORKSPACE/tests/e2e/configurations/micronaut_test_config.yaml --skip-deps || log_fail
+$RUN_MACARON analyze -purl pkg:maven/io.micronaut/[email protected] --skip-deps || log_fail
 
 for i in "${COMPARE_FILES[@]}"
 do
@@ -358,13 +359,14 @@ check_or_update_expected_output $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED ||
 echo -e "\n=================================================================================="
 echo "Run integration tests with configurations for FasterXML/jackson-databind..."
 echo -e "==================================================================================\n"
-JSON_RESULT=$WORKSPACE/output/reports/github_com/FasterXML/jackson-databind/jackson-databind.json
+JSON_RESULT=$WORKSPACE/output/reports/maven/com_fasterxml_jackson_core/jackson-databind/jackson-databind.json
 
 echo -e "\n----------------------------------------------------------------------------------"
 echo "FasterXML/jackson-databind: Check the e2e output JSON file with config and no dependency analyzing."
 echo -e "----------------------------------------------------------------------------------\n"
 JSON_EXPECTED=$WORKSPACE/tests/e2e/expected_results/jackson-databind/jackson-databind.json
-$RUN_MACARON analyze -c $WORKSPACE/tests/e2e/configurations/jackson_databind_config.yaml --skip-deps || log_fail
+$RUN_MACARON analyze -purl pkg:maven/com.fasterxml.jackson.core/[email protected] --skip-deps || log_fail
+# Original commit f0af53d085eb2aa9f7f6199846cc526068e09977 seems to be first included in version tagged commit 2.14.0-rc1.
 
 check_or_update_expected_output $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED || log_fail
 
@@ -373,7 +375,7 @@ check_or_update_expected_output $COMPARE_JSON_OUT $JSON_RESULT $JSON_EXPECTED ||
 # echo -e "----------------------------------------------------------------------------------\n"
 # DEP_EXPECTED=$WORKSPACE/tests/dependency_analyzer/expected_results/cyclonedx_FasterXML_jackson-databind.json
 # DEP_RESULT=$WORKSPACE/output/reports/github_com/FasterXML/jackson-databind/dependencies.json
-# $RUN_MACARON analyze -c $WORKSPACE/tests/dependency_analyzer/configurations/jackson_databind_config.yaml || log_fail
+# $RUN_MACARON analyze -purl pkg:maven/com.fasterxml.jackson.core/[email protected] || log_fail
 
 # check_or_update_expected_output $COMPARE_DEPS $DEP_RESULT $DEP_EXPECTED || log_fail
 
@@ -654,3 +656,14 @@ then
     echo -e "Expect zero status code but got $?."
     log_fail
 fi
+
+# Testing the Commit Finder's tag matching functionality.
+echo -e "\n----------------------------------------------------------------------------------"
+echo "Testing Commit Finder tag matching functionality."
+echo -e "----------------------------------------------------------------------------------\n"
+python $TEST_COMMIT_FINDER || log_fail
+if [ $? -ne 0 ];
+then
+    echo -e "Expect zero status code but got $?."
+    log_fail
+fi