Project 5: Richard Lee

README.md

@@ -3,26 +3,63 @@ WebGL Deferred Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Richard Lee
+* Tested on: Windows 7, i7-3720QM @ 2.60GHz 8GB, GT 650M 4GB (Personal Computer)
 
-### Live Online
+[![](img/thumb.png)](https://leerichard42.github.io/Project5-WebGL-Deferred-Shading-with-glTF/)
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+## Features
 
-### Demo Video/GIF
+* Basic Deferred Shading Pipeline with Lambert and Blinn-Phong shading
+* Lighting Scissor Test Optimization
+* Toon Shading
+* Compact Normal Buffer Optimization
+* Screen-Space Motion Blur
+* Variable Material Properties
 
-[![](img/video.png)](TODO)
+<img src="img/preview.gif" width="600">
 
-### (TODO: Your README)
+#### Basic Pipeline
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+This deferred shading renderer implements a deferred shader which lights a scene based on geometry buffers calculated in an initial pass. It uses the Lambert and Blinn-Phong reflection models with point lights to calculate the lighting for each light in the scene.
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+#### Toon Shading
 
+<img src="img/toon.gif" width="600">
+
+An alternate toon shading view was added, which clamped the lighting at fixed intervals to give a cartoon look. This effect had a negligible effect on performance, as it only consisted of a few additional lines in the deferred shading stage.
+
+#### Scissor Test
+
+<img src="img/scissor.gif" width="600">
+
+<img src="img/scissor_performance.png" width="500"><img src="img/scissor_cpu.png" width="500">
+
+The scissor test optimization calculates the screen space bounding box and creates a scissor for each light before performing the lighting calculations, which is very effective because the majority of lights only take up a small portion of the scene when rendered. As seen when observing the average time per frame with and without the scissor test, the scissor test provided a substantial improvement in performance as the number of lights increased, and also decreased in performance at a slower rate than without the optimization.
+
+When using the Firefox JavaScript profiler, the timing results did not take into account the WebGL draw calls, and only looked at the runtime on the CPU. This showed the scissor test as taking much longer than without the optimization, which was due to the fact that the screen space bounding box calculations were being performed for each light on each frame.
+
+In addition, there was a visual difference when enabling the optimization, as lights that were behind the camera did not register a bounding box on the screen, even if they had initially cast light past the camera.
+
+#### G-Buffer Optimization
+
+<img src="img/gbuffer.png" width="500">
+
+The number of g-buffers was reduced by packing the geometry and texture normals into a single buffer. This was done by only storing the x and y components for the normals, as the z component could be recalculated from these components when performing the shading calculations. Performance wise, there was not much difference in the time taken per render call on the CPU, as seen in the chart above - the time per frame also remained about the same with and without the optimization. However, since we were able to remove the use of an entire buffer, this optimization would definitely be effective in terms of memory usage.
+
+One caveat was that the recalculated normals seemed to have more contrast than without the optimization, giving a slightly different look to the model. This could be due to numerical precision issues, or a problem with the normal recalculation.
+
+#### Motion Blur
+
+<img src="img/noblur.gif" width="400"><img src="img/blur.gif" width="400">
+
+Screen-space motion blur was also implemented as a post-process shader, by calculating how far each visible point on the screen had moved relative to the camera using the camera matrix stored from the previous frame and the position g-buffer. This gave a screen-space velocity for each point, which was then used to interpolate the result from the deferred shading pass. This post-process did not have a noticeable effect on performance of the renderer.
+
+#### Material Properties
+
+<img src="img/materials.png" width="500">
+
+Additional variability in materials was included with the addition of a specular exponent value in the g-buffers, which allowed for multiple objects with different amounts of reflectivity to be rendered at the same time.
 
 ### Credits
 

glsl/copy.frag.glsl

@@ -3,18 +3,26 @@
 precision highp float;
 precision highp int;
 
+uniform bool u_packNormals;
 uniform sampler2D u_colmap;
 uniform sampler2D u_normap;
+uniform float u_specmap;
 
 varying vec3 v_position;
 varying vec3 v_normal;
 varying vec2 v_uv;
 
 void main() {
-    // TODO: copy values into gl_FragData[0], [1], etc.
     // You can use the GLSL texture2D function to access the textures using
     // the UV in v_uv.
 
-    // this gives you the idea
-    // gl_FragData[0] = vec4( v_position, 1.0 );
+     gl_FragData[0] = vec4( v_position, u_specmap );
+     gl_FragData[2] = texture2D(u_colmap, v_uv);
+     if (u_packNormals) {
+        gl_FragData[1] = vec4( v_normal.xy, texture2D(u_normap, v_uv).xy );
+     }
+     else {
+        gl_FragData[1] = vec4( v_normal, 1.0 );
+        gl_FragData[3] = texture2D(u_normap, v_uv);
+     }
 }

glsl/deferred/blinnphong-pointlight.frag.glsl

@@ -4,6 +4,9 @@ precision highp int;
 
 #define NUM_GBUFFERS 4
 
+uniform bool u_toon;
+uniform bool u_packNormals;
+uniform vec3 u_eyePos;
 uniform vec3 u_lightCol;
 uniform vec3 u_lightPos;
 uniform float u_lightRad;
@@ -12,6 +15,10 @@ uniform sampler2D u_depth;
 
 varying vec2 v_uv;
 
+vec3 reconstructNormal(vec2 xy) {
+    return vec3(xy, sqrt(1.0 - dot(xy, xy)));
+}
+
 vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
     normap = normap * 2.0 - 1.0;
     vec3 up = normalize(vec3(0.001, 1, 0.001));
@@ -20,20 +27,55 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
     return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor;
 }
 
+const float levels = 4.0;
+const float invLevels = 1.0 / levels;
+float applyToonFilter(float value) {
+    return floor(value * levels) * invLevels;
+}
+
 void main() {
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
     vec4 gb2 = texture2D(u_gbufs[2], v_uv);
     vec4 gb3 = texture2D(u_gbufs[3], v_uv);
     float depth = texture2D(u_depth, v_uv).x;
-    // TODO: Extract needed properties from the g-buffers into local variables
 
+    vec3 pos = gb0.xyz;     // World-space position
+    float specExp = gb0.w;
+    vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
+    vec3 geomnor, normap;
+    if (u_packNormals) {
+        geomnor = reconstructNormal(gb1.xy);  // Normals of the geometry as defined, without normal mapping
+        normap = reconstructNormal(gb1.zw);  // The raw normal map (normals relative to the surface they're on)
+    }
+    else {
+        geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
+        normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
+    }
+    vec3 nor = normalize(applyNormalMap (geomnor, normap));     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
     // If nothing was rendered to this pixel, set alpha to 0 so that the
     // postprocessing step can render the sky color.
     if (depth == 1.0) {
         gl_FragColor = vec4(0, 0, 0, 0);
         return;
     }
 
-    gl_FragColor = vec4(0, 0, 1, 1);  // TODO: perform lighting calculations
+    vec3 posToLight = u_lightPos - pos;
+    vec3 L = normalize(posToLight);
+    vec3 R = normalize(-reflect(L, nor));
+    vec3 E = normalize(u_eyePos - pos);
+
+    float diffuse = clamp(max(dot(nor, L), 0.0), 0.0, 1.0);
+    float specular = clamp(pow(max(dot(R, E), 0.0), specExp), 0.0, 1.0);
+
+    if (u_toon) {
+        diffuse = applyToonFilter(diffuse);
+        specular = applyToonFilter(specular);
+    }
+
+    vec3 diffuseCol = u_lightCol * colmap * diffuse;
+    vec3 specularCol = u_lightCol * vec3(1.0) * specular;
+
+    float attenuation = pow(max(0.0, u_lightRad - length(posToLight)) / u_lightRad, 0.5);
+    gl_FragColor = vec4(attenuation * (0.3 * diffuseCol + 0.7 * specularCol), 1.0);
 }

glsl/deferred/debug.frag.glsl

@@ -5,13 +5,18 @@ precision highp int;
 #define NUM_GBUFFERS 4
 
 uniform int u_debug;
+uniform bool u_packNormals;
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
 
 varying vec2 v_uv;
 
 const vec4 SKY_COLOR = vec4(0.66, 0.73, 1.0, 1.0);
 
+vec3 reconstructNormal(vec2 xy) {
+     return normalize(vec3(xy, sqrt(1.0 - dot(xy, xy))));
+ }
+
 vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
     normap = normap * 2.0 - 1.0;
     vec3 up = normalize(vec3(0.001, 1, 0.001));
@@ -26,27 +31,33 @@ void main() {
     vec4 gb2 = texture2D(u_gbufs[2], v_uv);
     vec4 gb3 = texture2D(u_gbufs[3], v_uv);
     float depth = texture2D(u_depth, v_uv).x;
-    // TODO: Extract needed properties from the g-buffers into local variables
-    // These definitions are suggested for starting out, but you will probably want to change them.
+
     vec3 pos = gb0.xyz;     // World-space position
-    vec3 geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
     vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
-    vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
-    vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
+    vec3 geomnor, normap;
+    if (u_packNormals) {
+        geomnor = reconstructNormal(gb1.xy);  // Normals of the geometry as defined, without normal mapping
+        normap = reconstructNormal(gb1.zw);  // The raw normal map (normals relative to the surface they're on)
+    }
+    else {
+        geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
+        normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
+    }
+    vec3 nor = normalize(applyNormalMap (geomnor, normap));     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
 
     // TODO: uncomment
     if (u_debug == 0) {
         gl_FragColor = vec4(vec3(depth), 1.0);
     } else if (u_debug == 1) {
-        // gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
+         gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
     } else if (u_debug == 2) {
-        // gl_FragColor = vec4(abs(geomnor), 1.0);
+         gl_FragColor = vec4(abs(geomnor), 1.0);
     } else if (u_debug == 3) {
-        // gl_FragColor = vec4(colmap, 1.0);
+         gl_FragColor = vec4(colmap, 1.0);
     } else if (u_debug == 4) {
-        // gl_FragColor = vec4(normap, 1.0);
+         gl_FragColor = vec4(normap, 1.0);
     } else if (u_debug == 5) {
-        // gl_FragColor = vec4(abs(nor), 1.0);
+         gl_FragColor = vec4(abs(nor), 1.0);
     } else {
         gl_FragColor = vec4(1, 0, 1, 1);
     }

glsl/post/one.frag.glsl

@@ -2,7 +2,10 @@
 precision highp float;
 precision highp int;
 
+uniform bool u_motion;
 uniform sampler2D u_color;
+uniform mat4 u_prevCameraMat;
+uniform sampler2D u_gbuf0;
 
 varying vec2 v_uv;
 
@@ -16,5 +19,24 @@ void main() {
         return;
     }
 
-    gl_FragColor = color;
+    if (u_motion) {
+        vec3 worldPos = texture2D(u_gbuf0, v_uv).xyz;
+        vec4 prevPos = u_prevCameraMat * vec4(worldPos, 1.0);
+        prevPos /= prevPos.w;
+        vec2 prev_uv = prevPos.xy * 0.5 + 0.5;
+        vec2 velocity = (v_uv - prev_uv) / 2.0;
+
+        float numSamples = 6.0;
+        vec2 uv = v_uv + velocity;
+        for (int i = 1; i < 6; ++i) {
+            color += texture2D(u_color, uv);
+            uv += velocity;
+        }
+
+        gl_FragColor = color / numSamples;
+    }
+    else {
+        gl_FragColor = color;
+    }
+
 }

glsl/red.frag.glsl

@@ -3,5 +3,5 @@ precision highp float;
 precision highp int;
 
 void main() {
-    gl_FragColor = vec4(1, 0, 0, 1);
+    gl_FragColor = vec4(1, 0, 0, 0.1);
 }