Lossless video streaming over USB

[eszdman]

Now headsets that support streaming have an increasingly faster USB controller, this allows the use of binning or similar techniques used by PC and headset GPU to transfer data without loss and higher latency.

Basically it should be several methods which you can select it's pretty cheap by computations on both headset and PC in compare to compression methods:

1. Pixel binning - combine nxm pixels into one and reconstruct it on headset shader code.
2. Pixel binning with pixel shift - similar method that use binning but shift input image every frame and reconstructs it on headset shader (similar to TAA).
3. Interlacing output - Can be used with previous methods, we can interlace frames in mosaic-like pattern to get higher frame rate with same baud rate.

With this techniques we can achieve insane amount of baud rate comparing to compression 960Mbps vs 20000+Mbps this will lead to a significant improvement in image quality, especially when the image is highly detailed.

I hope that my proposal will be passed on to the ALVR community because at the moment there are no such solutions.

[zmerp]

Thanks for the proposal.
Currently we are giving priority to other things. I believe you can achieve a good image quality using the current encoders given enough bitrate.

[eszdman]

I believe you can achieve a good image quality using the current encoders given enough bitrate.

No it's not, you can't change the fact that there's missing data with such compression schemes (it's aimed at working with low bitrates) comparing to pure data over display port / USB

[lividhen]

With the optics on the quest pro and 3 compression artifacts, even at high bitrates, are very clearly visible. I agree that current streaming methods are good enough for now but it might be a good idea to eventually look in to something like this.

[slxdy]

Hey, I'm not very experienced with video encoders and such, but is the 1gbps bit rate enough to stream at full res and refresh rate to the Quest 3? I've been trying to achieve the same quality and performance I get with the official meta link app, but I simply wasn't able to. I'm trying to stream native resolution and 120hz, but it stutters a LOT, to the point where using ALVR just isn't worth it. I want to switch to ALVR so I can play on Linux, but right now I'm unable to achieve the same quality/performance.

[eszdman]

Hey, I'm not very experienced with video encoders and such, but is the 1gbps bit rate enough to stream at full res and refresh rate to the Quest 3?

It depends on your content, but for a detailed image this may not be enough.
When you work with high bitrate and codec, you are limited by the encoding and decoding speed of the host and client, by removing the codec you can transmit a higher bitrate.
The amount of data without a codec can be calculated using a simple formula: S=2 * W * H * F * P — W,H - eye resolution, F - framerate, P - pixel size in bits (usually it's 3*8 bits). This will be the number of bits needed to transmit an image of a certain resolution. You can also use this formula to find out the compression ratio and understand how much worse the image can be.