You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Dec 31, 2020. It is now read-only.
Float10 and Float11 encoding of zero (and very small float values) in toFloat10 and toFloat11 functions of CompressorRGB.cpp produce and assertion in the nvDebugCheck for the exponent field. The E variable is of type unsigned int and very small values produce a negative value of E, which when cast to unsigned gives a number greater than the 32 limit in nvDebugCheck. In this state, setting a compressor output format to R11_G11_B10F has a high chance of producing the assertion and failing.
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Float10 and Float11 encoding of zero (and very small float values) in toFloat10 and toFloat11 functions of CompressorRGB.cpp produce and assertion in the nvDebugCheck for the exponent field. The E variable is of type unsigned int and very small values produce a negative value of E, which when cast to unsigned gives a number greater than the 32 limit in nvDebugCheck. In this state, setting a compressor output format to R11_G11_B10F has a high chance of producing the assertion and failing.
The text was updated successfully, but these errors were encountered: