fix: force half float in texture if it's exact and OES_texture_float_linear is not available #3117
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
The origin of the problem is this bug in OHIF, a consumer of vtk: OHIF/Viewers#4286
For certain mobile Android devices, instead of rendering a texture, the canvas was solid gray/black. It happens regardless of OS version/browser, but eventually I narrowed it down to devices that use Mali GPUs (which is likely because they don't support OES_texture_float_linear).
Since it was working everywhere previously and I had a good version of OHIF, I bisected until this first bad commit, which among other things, bumps the version of vtk.js from
29.3.0
to29.7.0
. After checking all the vtk versions in that range, I found that29.5.0
was the last good one, and29.5.1
was the first bad one with gray/black canvases. This issue has been present on master ever since.Luckily (or unluckily) this was a patch release that only included 2 commits. Of them, the relevant one is this.
The commit changed the texture format to sometimes use *32F versions, and added this warning:
The problem with using full floats is in webgl2 they aren't texture filterable without OES_texture_float_linear, while half floats are always filterable even without the extension. Using full floats triggers the warning/black textures.
I assume all of these are the same problem and would be fixed by this change:
Results
With this change, canvases are no longer black in OHIF and render the images again everywhere.
Changes
This change forces half floats in cases where:
I don't know if there's any way to solve this if the data doesn't fit in halfs, maybe with another option similar to preferSizeOverAccuracy.
PR and Code Checklist
npm run reformat
to have correctly formatted codeTesting
I only reproduced this on Androids with Mali GPUs. I would assume it happens on any device that supports webgl2 but not OES_texture_float_linear.
If you have such a device, you can see the problem here (a gray square is rendered, instead of an image of a brain CT): https://viewer.ohif.org/viewer?StudyInstanceUIDs=2.16.840.1.114362.1.11972228.22789312658.616067305.306.2
For testing the changes, the easiest way I know is to clone, compile and run OHIF, and configure resolutions in its package.json to use these changes in vtk:
Then open this: http://localhost:3000/viewer?StudyInstanceUIDs=2.16.840.1.114362.1.11972228.22789312658.616067305.306.2
With the changes enabled you should see the brain CT, without the changes you should not.