Remove support for preview image and thumbnail listfiles #1667
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #1029
Before launch, we implemented a set of text files that contained lists of preview images and thumbnails for each instrument. Originally these were designed to speed up calls to generate_preview_images, in that it could check the text file to see if a preview image already existed, and skip the creation of a new preview image if so.
But after launch, once the data volume increased enough, this method was no longer saving any time. So we removed the functionality that had the preview image generator open and read in the list files. However, we left in place the code that was adding names of new preview images to the list files at the end of each run of generate_preview_images.py. So the files continued to grow, but were not being used for anything. For NIRCam on production, the listfile is now over 400MB.
This PR removes all support for the listfiles. The files are no longer opened and have new entries added. This should speed up generate_preview_image.