Skip to content

Commit

Permalink
test sync efficiency
Browse files Browse the repository at this point in the history
use popen

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now

fix: generate only csv for now
enh: improve robustness of download progress
maximize the potential of s5cmd sync to avoid downloading the same data again

manifest validator function now also updates the input manifest with download directory

to make optimal use of s5cmd sync, a dry run is performed before the actual download,
the output from dry run sync is parsed to generate a synced manifest

download_dicom_series utilizes download_from_selection to make of sync logic
instead of duplicating the code

add additional endpoints download_dicom_studies, download_dicom_patients,
download_dicom_patients and download_collection, all of them routed to
download_from_selection
  • Loading branch information
vkt1414 committed Apr 1, 2024
1 parent 8e8121d commit 301aeb7
Showing 1 changed file with 224 additions and 125 deletions.
Loading

0 comments on commit 301aeb7

Please sign in to comment.