WaveSet Mesh Alpha Channel

[Hoikas]

PlasmaMax is stuffing... something... into the alpha channel of the vertex data automatically on export. See plMeshConverter.cpp SetWaterColor(). Figure out what's going on here and see if we need to be doing that as well.

[Jrius]

Vertex color alpha stores the length of the linked edges (kinda). In the end, it's just a multiplier for the final wave displacement in the vertex shader (so if alpha == 0, the vertex stays in place, and the vertex moves more freely with an alpha closer to 1).

I won't pretend to fully understand what SetWaterColor() does (usual undocumented mess from Cyan), but the gist of it is the wider the face (or the longer its edges), the lower the alpha (and thus geometric movement) of its linked vertices.

The waveset vertex shader is just a sum of sine functions of various frequencies (sortof). Like any function, it need enough samples to display correctly to avoid aliasing. In this case, the vertices are the samples. The problem is that 2^16 vertices per mesh isn't enough to display waves for wide bodies of water (and GPU tesselation wasn't an option). For this reason, artists usually put more vertices near the shore, and less towards the horizon. This way waves can still be seen near the avatar where they matter. This also makes the water cubemap more stable up close because its reflection vector is computed per-vertex and not per-pixel.

Problem is, the cubemap is still VERY unstable near the horizon due to the low mesh density and wide faces. So any wave geometric movement (which also recomputes the vertex normal) will make the cubemap wobble horribly. This is where you want a vertex alpha of zero. So as an artist, you would want to paint the alpha layer white near the shores where the avatar is standing, completely black further off (even a byte value of 1/255 is too much, you really want zero), and apply shades of grey in between.

I assume Cyan wanted to automate the process based solely on face area, but this is a bad idea IMHO. Their solution doesn't take the actual geostate frequency into account. Competent artists will figure it out better, and it also gives them the option to scale waves down in some areas to avoid clipping with other meshes.

A final note: if you want to actually control the transparency of the water, there are two ways to do this. The first is having the vertex' Z coordinate closer to zero (object origin), which also scales geometric movement down (and maybe other stuff that I don't remember). The second option is painting the vertex' red channel, since it behaves like the alpha channel usually does (Plasma just keeps doing things backwards :P ).
Also, blue channel is fresnel, and it appears green is unused.

[dpogue]

From GPU Gems:

The red component governs the overall transparency, making the water surface completely transparent when red goes to zero. Green modulates the strength of the reflection on the surface, making the water surface matte when green is zero. Blue limits the opacity attenuation based on viewing angle, which affects the Fresnel term.

---

@colincornaby Since you were writing the Metal shader for wavesets recently, does the above description of how the alpha vertex colour channel is used seem correct?

[Jrius]

The shader I'm looking at is an old Unity port of the GPUgems one, so it's possible I messed up on the blue/green channels. There might also be small differences between the Plasma and GPUgems version 🤷

[colincornaby]

Lemme review the Metal code. The Metal code is still in a half-assembly-ish-C++ state, so deriving meaning from what each of the operations means is still tough. But - generally the wave shaders do piggyback a lot of information in things like color channels. HLSL 1.1 assembly doesn't have open ended variables between the vertex and pixel shader stages, so it's really common in those shaders to stuff things inside color registers.

[dpogue]

There might also be small differences between the Plasma and GPUgems version

The GPU Gems text makes no mention of the alpha channel, so let's assume there are differences

[colincornaby]

From the Metal source:

The filter is used in later stages for calculating the waveform.

If Cyan needed to pass in per vertex data this probably would be the way to do it. I don't think HLSL 1.1 allowed for open ended vertex buffers that could have other anonymous information in there. They would have had to have shoved it in something like a color attribute/register.

[colincornaby]

Oh, here you go. This comment is very helpful:

    // v5.r = overall transparency
    // v5.g = reflection strength (transparency)
    // v5.b = overall wave scaling
    //
    // v5.a is:
    // v5.w = 1/(2.f * edge length)
    // So per wave filtering is:
    // min(max( (waveLen * v5.wwww) - 1), 0), 1.f);
    // So a wave effect starts dying out when the wave is 4 times the sampling frequency,
    // and is completely filtered at 2 times sampling frequency.

    // We'd like to make this autocalculated based on the depth of the water.
    // The frequency filtering (v5.w) still needs to be calculated offline, because
    // it's dependent on edge length, but the first 3 filterings can be calculated
    // based on this vertex.
    // Basically, we want the transparency, reflection strength, and wave scaling
    // to go to zero as the water depth goes to zero. Linear falloffs are as good
    // a place to start as any.
    //
    // depth = waterlevel - r6.z        => depth in feet (may be negative)
    // depthNorm = depth / depthFalloff => zero at watertable, one at depthFalloff beneath
    // atten = minAtten + depthNorm * (maxAtten - minAtten);
    // These are all vector ops.
    // This provides separate ramp ups for each of the channels (they reach full unfiltered
    // values at different depths), but doesn't provide separate controls for where they
    // go to zero (they all go to zero at zero depth). For that we need an offset. An offset
    // in feet (depth) is probably the most intuitive. So that changes the first calculation
    // of depth to:
    // depth = waterlevel - r6.z + offset
    //      = (waterlevel + offset) - r6.z
    // And since we only need offsets for 3 channels, we can make the waterlevel constant
    // waterlevel[chan] = watertableheight + offset[chan],
    // with waterlevel.w = watertableheight.
    //
    // So:
    //  c25 = waterlevel + offset
    //  c26 = (maxAtten - minAtten) / depthFalloff
    //  c27 = minAtten.
    // And in particular:
    //  c25.w = waterlevel
    //  c26.w = 1.f;
    //  c27.w = 0;
    // So r4.w is the depth of this vertex in feet.

    // Dot our position with our direction vectors.`

[Jrius]

Ah, thanks for the clarification, this is much clearer now.
Yes, so assuming everything works as described, then all 4 geometric waves are correctly scaled down to zero when their frequency is higher than the number of vertices available, which should be perfect. Vertex alpha also stores the inverse of edge length so the low precision shouldn't cause issues.
I'm still not quite sure why SetWaterColor() relied on face area at some point instead of just edge length, or what kSmooth does. But from what I can understand it should do the job correctly.
I'm a bit concerted about this though:

float4 filter = inColor.wwww * uniforms.Lengths;
filter = max(filter, uniforms.NumericConsts.xxxx);
filter = min(filter, uniforms.NumericConsts.zzzz);

Shouldn't 1 be subtracted before clamping, so that the waves truly die ? The comment in your second post says wave filtering is min(max( (waveLen * v5.wwww) - 1), 0), 1.f) (It's late here so maybe I'm forgetting something.)

Looks like implementing this sounds worthwile, although low-priority. I still like the idea of having artisting control over wave amplitude, but maybe the vertex' height can work for that purpose...

[dpogue]

I still like the idea of having artisting control over wave amplitude, but maybe the vertex' height can work for that purpose...

I was just wondering earlier what would happen if you added bone animations to a waveset mesh... 😇

[colincornaby]

@Jrius It's a good question - but... here is the original HLSL assembly.

mul         r11, v5.wwww, c24;
max         r11, r11, c16.xxxx;
min         r11, r11, c16.zzzz;

For reference, the original HLSL assembly of the vertex shader can be found here:
https://github.com/colincornaby/Plasma/blob/master/Sources/Plasma/PubUtilLib/plSurface/ShaderSrc/vs_WaveFixedFin7.inl

The half-rewritten-into-C++ Metal version is here:
https://github.com/colincornaby/Plasma/blob/apple-metal-resync/Sources/Plasma/FeatureLib/pfMetalPipeline/ShaderSrc/WaveSet7.metal

(Always happy to have more eyes on this to get the Metal shader C++ version more clear. It would be helpful in future GLSL or newer HLSL ports.)

[Hoikas]

If I'm reading the discussion correctly, then it sounds like having Korman generate this edge length value and stuffing it into the vertex alpha channel is desirable, assuming the Age creator has not already manually painted vertex alpha onto the waveset.

[Jrius]

I was just wondering earlier what would happen if you added bone animations to a waveset mesh... innocent

I'm not sure in which situation this would be useful... but people always find uses for crazy ideas :D

It's a good question - but... here is the original HLSL assembly.

I'm not blaming your Metal conversion, mind you, I'm blaming Cyan's assembly code. Today I had a deeper look, and it is indeed wrong like I suspected.
The earlier comment states:

// v5.w = 1/(2.f * edge length)
// So per wave filtering is:
// min(max( (waveLen * v5.wwww) - 1), 0), 1.f);
// So a wave effect starts dying out when the wave is 4 times the sampling frequency,
// and is completely filtered at 2 times sampling frequency.

This makes sense since we want the waves to completely die out when the sampling frequency is too low.

But in practice (since c16.xz == float2(0, 1)) the source assembly code translates to r11 = clamp01(v5.wwww * c24). It never subtracts 1.
Which means the waves start fading out at 2 times sampling frequency (too late), and never completely die out - so the cubemap will still be wobbly even from a distance.

Instead we want r11 = clamp01(v5.wwww * c24 - 1), so the correct assembly should have been:

mul         r11, v5.wwww, c24;
sub         r11, r11, c16.zzzz; // subtract 1 before clamping
max         r11, r11, c16.xxxx;
min         r11, r11, c16.zzzz;

---

Oh, and also worth noting. The comment says:
// v5.w = 1/(2.f * edge length)
In plMeshConverter::SetWaterColor(), kNumLens is set to 4 and not 2. If I'm right, this means waves should start dying out at 8 times sampling frequency, and be fully filtered at 4 times sampling frequency. I would say this would look a bit better if the assembly code were correct. Which it isn't - in the current version of the engine, this means waves will start dying out at 4 times sampling frequency (but again, they will never completely fade out).

TL;DR: the current version of Plasma's waveset shader has a bug, and working around it is going to be annoying.

---

So, now, what to do...

• Implementing plMeshConverter::SetWaterColor() in Korman is desirable, but only when exporting to MOULa once its shader bug is fixed.
• This means Ages exported to CC will still have cubemap wobblyness towards the horizon by default.
• CC Ages can be fixed manually by the artist by painting an alpha channel on the water mesh (with the caveat that it's not a perfectly linear modifier for vertex movement, but in my experience it's good enough). I'm not sure how many Age builders will bother though.
• We could also let artists supply their own vertex alpha on MOULa (in which case it would multiply the result of plMeshConverter::SetWaterColor()). This is just for finer control. Alternatively, the artist can just change the vertex position, which has a similar effect (although it also changes the final water transparency).

With that said, this is all just to fix a small visual glitch. I would say it's very low priority...

[colincornaby]

I'm not blaming your Metal conversion, mind you, I'm blaming Cyan's assembly code. Today I had a deeper look, and it is indeed wrong like I suspected.

Oh certainly. The MSL code was written to reproduce the HLSL code exactly. I didn't want to produce different results by renderer, which could then make ages look different by platform. Players and creators are going to need to trust that things look the same across all the renderers.

The downside is I've found multiple instances where I'm pretty sure the shaders diverge from GPU Gems or the comments. Which has led to a more literal translation of the assembly to try to sidestep that.

Changing these shaders is probably a wider discussion. It would change the appearance of content. As an OpenGL renderer comes online, that adds another synchronization point for changes.

Metal really prefers to precompile its shaders to bitcode at compile time. And Metal 3.0 will actually compile shaders to machine code at installation time. So I don't want to go down a path of having a real time shader generator in the client which would bypass that. But if we are going to start doing more work on the shaders, it might be helpful to get them into modern HLSL and try a tool like Shader Conductor. I've never used Shader Conductor, so I'd want to do an evaluation pass on it first. But it could help us keep changes in sync.

Anyway, this is probably getting really off topic. Basically: The shaders do diverge from what they should be doing for unknown reasons, it's made the Metal translation more messy and less optimal, and it would be great if we could reconcile it all in the future.