Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transfer - Fix delayed artifacts mechanism #961

Closed
wants to merge 2 commits into from

Conversation

yahavi
Copy link
Member

@yahavi yahavi commented Sep 13, 2023

  • All tests passed. If this feature is not already covered by the tests, I added new tests.
  • All static analysis checks passed.
  • This pull request is on the dev branch.
  • I used gofmt for formatting the code before submitting the pull request.

Tests PR: jfrog/jfrog-cli#2204

Enhance and optimize the following elements:

  • Delaying pom.xml files, not just *.pom.
  • Eliminating the need for recursion in delaying artifacts. Instead, transfer them in a sorted manner. This approach is significantly more efficient, especially considering that in Docker and Conan repositories, we triggered 2-3 invocations of doTransferWithProducerConsumer.

@yahavi yahavi force-pushed the fix-delayed-artifacts branch from 7192e3a to bc70c15 Compare September 13, 2023 15:50
@yahavi yahavi self-assigned this Sep 13, 2023
@yahavi yahavi added the improvement Automatically generated release notes label Sep 13, 2023
@yahavi yahavi marked this pull request as ready for review September 13, 2023 16:05
log.Info(fmt.Sprintf("Uploading %d delayed artifacts...", len(allDelayedArtifacts)))
allDelayedArtifacts = sortDelayedArtifacts(&allDelayedArtifacts, delayHelper)
// Use an empty delayUploadHelper to prevent further delays of these files.
shouldStop, err := uploadByChunks(allDelayedArtifacts, uploadChunkChan, base, delayUploadHelper{}, errorsChannelMng, pcWrapper)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a delayed artifact fails here, for example, 'conanfile.py', then the following file at the ordered delayed list is for example 'conaninfo.txt' and no issue at upload, won't it be a problem later when we try to upload It again?

@yahavi yahavi closed this Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Automatically generated release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants