Actually, there seems to be a lot of confusion about how to filter in log space when outputting linear depth, and why you need to. Let me clear things up.

There are two approaches to exponential shadow mapping:

1. Output exponential light depth map, prefilter linearly.

You can create a depth map for the light by outputting exp(depth) and then perform the typical gaussian/box/tent prefiltering as normal. In this case, when drawing the light, the ESM filtering is done like so:

float occluder = tex2D(shadowMap, texCoords);
float lit = occluder / exp(c*reciever);

The advantage of this method is that the prefiltering is less expensive than the next method. Hardware bilinear, anisotropic and mip filtering all automatically filter the shadow map correctly.

The disadvantage of this method is that it is precision hungry, because exp(depth) varies quickly. A 32 bit floating point texture allows a value of c=88 before overflow errors start to occur. 16 bits is not enough precision with this method in my experience as contact light leaking is very problematic.

2. Output linear depth, prefilter in log space.

You can create a depth map for the light by outputting depth as is, and then perform a gaussian/box/tent prefilter in log space. In this case, the when drawing the light, the ESM filtering is done like so:

float occluder = tex2D(shadowMap, texCoords);
float lit = exp(c*(occluder - reciever));

The advantages of this method:

• Less precision is required to store depth, which now varies linearly. So we only need 16 bits to store depth!
• The ESM filtering when drawing the light is slightly faster as we can remove a division.

The disadvantages:

• Hardware bilinear and anisotropic filtering will introduce some error, although it is generally close enough — the artifact is just a little bit of shadow overdarkening.
• Prefiltering must be done in log space, which is slower (see below).
• If mipmaps are used, they must be generated using filtering in log space as well.

Overall, these two methods represent a tradeoff between memory and ALU, with method 1 requiring more memory and less ALU overall.

So how and why do we filter in log space?

Consider for example that we wish to average two values in the shadow map (like in a 2×2 separable gaussian or box blur). Then we are averaging the two values exp(d1) and exp(d2).

Using gaussian/box blur weights w1 and w2, we then have:

w1*exp(d1) + w2*exp(d2)

exp(d1) * (w1 + w2*exp(d2 – d1))

exp(d1) * exp(log(w1 + w2*(exp(d2-d1))))

exp(d2 + log(w1 + w2*exp(d2-d1)))

So now the sum of the two exponentials is written as one exponential. Taking the log of the previous statement, we can perform the averaging of the box/gaussian blur by working on the exponentials argument only:

d2 + log(w1 + w2*exp(d2-d1))

So we filter the arguments of the exponentials, and then go back to exponential space when actually drawing the light, using exp(c*(occluder – reciever)) as we saw earlier.

I’ve generalized the above reasoning for two arguments to arbitrarily many arguments in the following code that can perform a box blur in log space, using an HLSL pixel shader. To perform a Gaussian blur instead, just replace the constant sample weights and 1.0 with the appropriate Gaussian weights.

sampler TextureSampler : register(s0);
#define SAMPLE_COUNT 3
float2 Offsets[SAMPLE_COUNT];

float log_space(float w0, float d1, float w1, float d2){
	return (d1 + log(w0 + (w1 * exp(d2 - d1))));
}

float4 Blur(float2 texCoord : TEXCOORD0) : COLOR0
{
	float v, B, B2;
	float w = (1.0/SAMPLE_COUNT);

	B = tex2D(TextureSampler, texCoord + Offsets[0]);
	B2 = tex2D(TextureSampler, texCoord + Offsets[0]);
	v = log_conv(w, B, w, B2);

	for(int i = 2; i < SAMPLE_COUNT; i++)
	{
		B = tex2D(TextureSampler, texCoord + Offsets[i]);
		v = log_conv(1.0, v, w, B);
	}

	return v;
}

So what does all this extra work get us? The error introduced by filtering linearly instead of in log space was so small in my tests, that I couldn’t produce a screenshot that clearly demonstrates it.

Here is the thread that inspired me to try filtering in log space, and also a clear difference between linear and log filtering is demonstrated.

I found that on the Xbox 360, log filtering didn’t cost anything extra, as the prefiltering step was texture bandwidth bound anyway — so I’ll leave it in place for now.

Raytraced render of the new test model.

This model (which is two copies of a sort of church-like building), casts shadows on itself (where we will see contact light leaking), and the windows provide various sized gaps in the self shadowing (where we will see ESM’s oversized shadows drown out the gaps).

Here is a render of the model(s) in the game engine, viewed from the inside:

1 tap ESM, 1024x1024 shadow map with 2 cascades, 4xMSAA + 3x3 box blur

Notice how light leaks through at the corners of the church in the image above.

By rescaling the ESM results further (and throwing away more of the blur), we can reduce this artifact:

However this makes the shadows even larger (“fatter”), and reduces the size that a window can be before light shows through. (The two small windows are already missing in the shadow.)

1 tap ESM with 1024x1024 shadow map, 2 cascades, 4xMSAA + 3x3 box blur

Notice the lack of shadow at the top of the church, near roof edge. Overall I would say that this is not too noticeable. However, this is a best case, where the church is quite large relative to the size of the scene (the ground edge in the background stops just before the depth range ends).

4 tap dithered PCF, 1024x1024 shadow map, 2 cascades.

When using 4 tap dithered PCF, there is no leaking and the small windows let some light through. Notice that the large windows on the right also now allow light through. Of course, the shadow edges are not as soft, and the shadows shimmer during movement, and the dithering pattern changes during camera movement.

4 tap dithered PCF

With 4 tap dithered PCF, self shadowing is improved. Notice that the roof now begins to self shadow the building. Also notice slight peter-panning in the roof shadows due to the roof offset being large, as slope-scale biasing must add more bias to the roof than other parts of the building (because the roof is at a large angle to the light).

4 tap PCF with 50px sampling area random disc

Above the shadows error is unstructured compared to the regular dithering pattern, and the sampling area is larger so the shadows appear softer. This sampling method mostly hides the underlying shadow map, although the dithering pattern changes during camera movement. It’s far from ideal, but may be better in some cases, where self shadowing is important, and contact light leaking is important. Adding two more cascades would sharpen the shadow edges while completely hiding the underlying shadow map.

Exponential Variance Shadow Mapping also solves these issues at 4x the memory usage, while still producing smooth shadows. I may experiment with EVSM, although I doubt that it is an affordable technique on the Xbox 360 unless significant compromises are made.

4 tap PCF with random sampling from 50px area disc

In the image above, we can see that there are shadow acne problems due to too little bias while at the same time the top roof shadows are peter-panning due to too much bias. Slope-scale bias is already implemented here, and only surfaces facing the light are drawn. Using more cascades, or a larger shadow map, or a smaller sampling area would solve this problem.

Final decisions on which shadow technique combinations will be used will be made when the scene is more complete, and left over performance is more limited.

1 tap ESM filtering, 2 cascades encoded into a 512x512 shadow map, and 5x5 box blur prefiltering of shadow map. Shadow map is rendered using 4xMSAA.

About 1/3 of the depth range is represented by the first cascade (and 2/3 in the second cascade). The difference in shadow size between these two cascades is much bigger than expected.

To reduce the artifacts, we can linearly interpolate between the cascades. This article suggests this method (look at figure 9). A region where blending will be done is marked below in light red.

Blend region where shadow cascades will be linearly interpolated.

I use a conditional shader branch so that samples from both cascades are only taken in the blend region. This means that reducing the blend region in screen space will reduce computation time. Two ways to reduce the blend region are:

• Reduce the blend region width.
• Increase the cascade split distance from the camera.

Note that increasing the cascade split distance will also reduce the resolution difference between cascades, and therefore make the cascade discontinuity smaller.

Blending between shadow cascades is performed on the previously shown blend region. Blending artifacts are highlighted with green arrows.

Click the above image to enlarge it. The middle green arrow highlights a particularly bad case, where a very long shadow is in the blend region. Increasing the blend region would make the artifact larger, but less noticeable.

Without modifying the cascades, blending between cascades can result in seams at the blend region:

A seam results from selection between both cascades.

Thankfully though, these seams can be avoided by making the cascades overlap (i.e their depth bounds should overlap with one another).

Edit: I demonstrated some harder cases for the important filters here. I also used a brighter light so that it’s easier to see the shadow edges

Over the past two weeks I’ve been working on various parts of my game engine’s shadow mapping, mostly experimenting with different shadow filtering techniques.

No shadow filtering.

No shadow filtering.

For reference, above are two images of a simple test scene without any shadow map filtering applied.

I started with basic 4 tap PCF, using a simple averaging of the samples. See this filtering below.

4 tap PCF taking the mean of the samples.

Then I moved on to edge-tap smoothing of the 4 tap PCF filtering method:

4 tap PCF with edge tap smoothing (a linear blending between sample values).

Then I experimented with using more than 4 samples:

7 tap PCF with edge tap smoothing.

Below is an implementation of the dithering idea in the article Shadow Map Antialiasing from GPU Gems 1.

4 tap PCF with screenspace dithering.

The GPU Gems 1 method above selects shadow sampling positions based on the fragment’s current screenspace coordinates. So I tried rotating the 4 samples based on the current fragment position (in screen space) instead. See the image below.

4 tap PCF with screenspace based rotations.

8 tap PCF using rotated poisson disc with screenspace based rotations.

I also tried doing the rotations based on object space instead:

4 tap PCF with object space based rotations.

Then I tried using a small lookup texture (32×32) with random 2×2 matrix rotations, used to offset the 4 samples in random directions based on a screenspace sampling of the random noise texture. See below.

4 tap PCF with 202 pixel sampling area, sampling chosen by taking rotations from a lookup texture.

4 tap PCF with 50 pixel sampling area, sampling chosen by taking rotations from a lookup texture.

4 tap PCF with 50 pixel sampling area, sampling chosen by taking rotations from a lookup texture.

It’s worth noting that even though the last two methods shown are the same and both use 4 samples, the first method chooses samples from a larger area than the second. This makes the penumbra larger, hides the underlying shadow map better, but also costs more texture bandwidth due to worse cache coherency. The 202 sampling area PCF requires a large bias, to the point where even with slope-scale biasing, I couldn’t choose a small enough bias to avoid shadow peter-panning completely.

The last method shown actually produces pretty good looking shadows, and is quite efficient. It also allows for an easily modifiable sampling area, which could be very useful — more on this later.

None of these filtering methods produced particularly smooth looking static shadows. Except for the last method displayed, all of these shadows “shimmer” horribly when the camera or light source moves. The rotated disc method shadows also shimmer, but the effect is not nearly as bad as the other PCF sampling methods. And even though they are all pretty fast, they require fairly large shadow maps. All of the images so far have used 2048×2048 shadow maps.

Next I started implementing exponential shadow maps (ESMs). These allow us to blur the shadow map or otherwise filter it before drawing the light, and then we can take a single sample from the shadow map, using an approximation of the shadow penumbra. This method avoids biasing problems that large penumbra PCF sampling has. It also requires less texture bandwidth.

I started by using 2xMSAA when creating the shadow map (which effectively blurs the shadow map edges). A 2048×2048 shadow map is used, and 2xMSAA doubles the size of the texture at creation time. This makes this texture 32MB in total, which is far larger than the Xbox 360′s 10MB eDRAM, and therefore far too large to be practical on the Xbox 360. A 5×5 seperable gaussian blur is also applied to the shadow map before drawing the light.

1 tap ESM with a small shadow map and 4xMSAA + 5x5 gaussian blur.

I also tried taking 4 samples of the shadow map, creating the shadow map with 4xMSAA, and not even bothering with a prefiltering step (so there is no gaussian blurring of the shadow map). This is potentially a faster method, and is the method that I am currently using for point light shadows (which are less dominant lights than the directional light shown in all of the screenshots so far).

4 tap ESM filtering with 4xMSAA used when creating shadow map. No shadow map prefiltering.

Since the results of the ESM with a gaussian blur prefilter are quite good, I started reducing the shadow map size. Below is a 512×512 shadow map with 4xMSAA (which means that the texture easily fits into the XBox EDRAM and therefore 4xMSAA incurs very little texture bandwidth cost). A 5×5 separable gaussian blur is applied so that a single sample is enough to produce a smooth penumbra.

1 tap ESM filtering, 2xMSAA + 5x5 gaussian prefiltering of shadow map.

This technique is fast and looks good. One problem with it though, is that the penumbra size is fixed, which is not realistic. In real life, light is scattered around, and light sources are not ideal points or directional lights. Percentage closer soft shadow (PCSS) approximate real life shadow penumbras by determining where the penumbra should be sharper or softer. Shadow penumbras are typically sharper close to the occluding object, and softer further from the object — instead of soft everywhere like in the image above.

The problem with a pre-filtered shadow map using one ESM tap, is that you cannot change the size of the penumbra by expanding the sampling kernel size (as there is only 1 sample) and the shadow map prefiltering used a fixed size kernel (a 5×5 Gaussian in this case). This is something that is easier with PCF, but as we saw, PCF does not look good with only a few samples. PCSS can be expensive, and is only an approximation — and not a very good one when there are multiple layered occluders. So this is a problem that I’ll defer until later. Possible solutions include varying the ESM exponent to vary the penumbra size, or combining PCF sampling techniques shown earlier with ESM to vary the kernel size.

Here’s a quick glimpse at what point light shadows are currently looking like with a deferred rendering setup, using a small 7 tap PCF filter and a 512×512 shadow map. The model used is Crytek’s Sponza model.

Point light shadows using dual paraboloid shadow mapping.

I wanted to save some texture bandwidth, so I tried to put one shadow map in the R channel of a R16G16 texture, and then the other shadow map in the G channel. However, you cannot use ColorWriteChannels on a non-blendable texture format in XNA (and HalfSingle textures are not blendable).

So I tried using two channels of a A2R10G10B10 texture instead (i.e. HDRBlendable on the Xbox). This worked, except that it didn’t give me the bandwidth savings I was hoping for.  It turns out that on the xbox, this format is stored as 32bpp in the EDRAM, but actually resolves to HalfVector4 (R16G16B16A16), 64bpp, when copying to system memory. So I was using the same amount of texture bandwidth in the end anyway, because the HDRBlendable format expands to twice the size on the xbox on resolve.

Instead, I’m experimenting with packing two float values into 32bpp 8bit/channel texture. The cons are:

• Packing/unpacking costs time in the pixel shader.
• Must switch blend state to mask the color channel when switching to drawing the next shadow map.
• Lower precision (although this is not a problem at all if the point lights are small enough, and the fixed decimal point place is chosen well).

The pros are:

• Half the texture bandwidth used to resolve the shadow map from the EDRAM to system memory.
• Avoid having to switch render targets when drawing the next shadow map.
• Save a little bit of texture bandwidth when reading from the shadow map (when drawing the light), because some lookups will use both channels at the same time. Mostly the R channel will have totally separate lookup locations from the G channel since the channels represent opposite hemispheres, but in some places (along the seams?) caching will save a little bit of bandwidth.

Whether using two textures or one texture with packing is better depends mostly on how much of a bottleneck texture bandwidth is vs fill rate.

There is yet another option.

For small shadow casting point lights, 8 bits of depth precision might be enough. Then up to four shadow maps can be stored in a single texture, without any extra encoding ALU costs. This might allow for many small shadow casting point lights.

Maximum of 7 lights touching each chunk in the terrain shader, but chunks were far too large due to CPU limitations on the XBOX.

To this end, I’ve been exploring some deferred lighting options.

For the Xbox 360, an attractive option is light-pre-pass, which suits the Xbox much moreso than a fully deferred setup, as LPP uses less intermediate texture buffering, and therefore avoids Xbox 360 texture bandwidth limitations, and avoids predicated tiling issues.

J. Coluna has a good LPP example, and he was wondering how to deal with the problem of being limited to 32 bit render target formats on the Xbox 360, yet wanting to store specular light data in the light buffer (for which I use HDRBlendable, on the xbox this format stores 10 bits for RGB and 2 bits for alpha).

I’ve put in somewhere between 500-900 hours into this project (schoolwork being mixed in there makes it hard to estimate), and I think that I’ve been very productive in that time. The project is now 8300 lines of code, in about 187 class files.

So, why didn’t I finish in time for DBP? Here are some of the mistakes I made:

• Overambitious project goals.
• Picked wrong light rendering design for the design of this game.
• Underestimated the complexity of implementing some design ideas, such as the terrain, and the lighting set up.
• Focused too much on building clean, pretty code, instead of hacking together a finished game.
• Spent too much time building the game engine, and too little time building the actual game logic.
• Spent two weeks building my own physics engine from scratch, instead of using an existing engine.

Here’s a composite of a few screenshots of my Git commit history:

(Click to enlarge.)

I tried to choose a game concept that was as simple as possible, while still maintaining some of the key elements from my 100+ page outline of a much more complex game.

That said, I was still far too ambitious for a 3 month project. I think that any 3D game that can be competitive with a 2D game is going to take a lot of extra time to polish well. For example, I didn’t realize how much time triplanar normal mapping would take to get right. It’s simple in theory, but getting the slope angles just right, getting the transitions between texturing planes just right, mapping the normal maps onto the correct plane, etc, etc, all took a lot of extra time.

Before deciding to do this project in first person 3D, I read that many people avoid indie 3D games because creating the artwork takes so much longer (or they can’t do it themselves). As one example, it took me about 45 minutes to create the doll enemy model (pretty quick, no?). I figured, well, I can create a 3D model of an enemy in ~45 minutes, so 3D artwork is not a big deal — I’ll just go make the next Halo now!

In reality, the artwork is no big deal. It’s really everything but the artwork that makes 3D much more work than 2D, especially from a first person perspective. When I say “artwork” I mean 3D models, textures, texture mapping, animations.

In a first person perspective, you can see thousands of meters into the distance, but also if the player looks straight down at the ground, since only a 1-2 meters are viewable, we expect to see a great deal of detail. Managing detail at close and far distances well is an order of magnitude harder than managing detail in a 2D game. In a 2D game, or even in a 3D game with a relatively fixed camera perspective (e.g. Starcraft or Diablo), you know how much geometry can be viewable in the worst case, and there’s much less detail scaling needed.

As an example of detail scaling, getting terrain detail to scale well in this game was done over the course of many weeks. Indeed, it’s still not perfect. If you’ve tried your hand at realtime 3D drawing, then you’ve probably done a terrain. However, you may not have encountered the mess of fading textures and specular/normal/emmissive maps between detail levels. I used geometry instancing for the distant chunks, which meant no lights in the distance in my lighting model. Things like this added up quickly.

I thought a bit about what the title of this post should be. A reference to Duke Nukem Forever felt appropriate at first, until I realized that it’s probably not fair to compare my 3 months of development to 14 years of off-and-on DNF development. Hell, even if I am writing DNF-level vaporware, at least Duke Nukem Forever is finally being released, right?

This is Morbo, but that's who I think of whenever I see Norbo's name. Also I like pictures in my posts.

The following code creates 392 spheres. I replaced the code in the BEPU terrain demo that adds boxes to the terrain with this code, to do a quick benchmark of spheres on a terrain.

Random rand = new Random();

for (int i = 0; i < 7; i++)
{
    for (int j = 0; j < 7; j++)
    {
        for (int k = 0; k < 8; k++)
        {
            Vector3 position =
                new Vector3((float) rand.NextDouble()*10 + i*128,
                    400 + -j*2,
                    (float) rand.NextDouble()*10 + k*128);
            float radius = (float) rand.NextDouble() + 1;
            Space.Add(new Sphere(position,
                radius,
                radius * radius * radius));
        }
    }
}

First, I want to point out that BEPU is an awesome project, and is generally very efficient from what I’ve been able to measure.

My physics simulation is much, much more limited in scope compare to BEPU. I don’t support any type of constraints, and I don’t have accurate restitution or friction.

In the BEPU terrain demo, on my E5300 2.6Ghz, overclocked to 3.0Ghz, on a single thread, the 392 previously mentioned spheres start the simulation updates off at about 1-2ms (when all spheres are in the air) until all of the spheres hit the terrain at which point the simulation takes 7-17ms per update, until all of the spheres reach a resting (inactive) state, where the simulation takes 2ms per update.

On the XBOX 360, this same simulation starts at 7ms per update, and then as soon as the spheres hit the terrain, the simulation jumps to 60ms per update, and steadily increases to well over 100ms per update over the course of the next 1-2 minutes, until it eventually starts to speed back up as spheres start to become inactive.

Edit: Testing with a much flatter terrain resulted in update times of around 30ms per frame with the default setup. Excellent! BEPU totally wins then.

Under similar conditions in my game (so the terrain is a different shape), my simulation takes 30ms per update on the XBOX 360, when all 400 spheres are in the air, falling at high speed under gravity. It starts slow compared to BEPU because I always sphere-cast spheres against the terrain no matter what, since they usually will intersect the terrain in the case of this game. The simulation then continues to about 35ms when the spheres hit the terrain, and updates continue to take about 30-35ms as the spheres eventually slow down.

Edit: some simple optimizations reduced this time to 21ms.

Of this 35 ms, the breakdown is:

• The sweep and prune broadphase takes 3ms per update.
• The sphere-terrain contact point detection/generation takes 7ms per update, and the sphere-sphere contact point generation takes 23ms per update.
• The collision solver takes a total of about 3ms

Implementing a resting/inactive state would greatly reduce the 31ms of contact point generation after 30-60 seconds into the simulation, as most of the spheres would no longer need to generate a new contact points unless they started moving again. Implementing this optimization is not a priority right now, but I’m very curious to see what can be done with that.

In my simulation, 7 contact points (this is somewhat arbitrary) are generated for every sphere with the terrain no matter what position and velocity the sphere has. This is a place for some optimization if desired.

I generate a completely new set of contacts every single update. I think BEPU might persist contacts across multiple frames. I’m not sure if it’s practical for me to do this or not — I haven’t given it much thought yet. I’m also not sure how this affects BEPU performance.

A screenshot of the physics simulation running on the XBOX 360, while the white spheres are still rolling around on the terrain.

In the end, the difference between BEPU and my physics for this somewhat unscientific preliminary comparison seems to indicate that BEPU is about 3x slower for 400 active spheres colliding with a terrain at not very high speed. (In my simulation ~1m radius spheres travelled at speeds in the range of 1m/s – 20m/s).

Again, my simulation is limited in what it can simulate compared to BEPU, but it looks like in the special case of my game, where I want many spheres with only sphere-sphere and sphere-terrain contact, and I don’t care about accurate rotation and friction simulation, it looks like my physics simulator might make more sense.

I’m going to do some more tests, and get in touch with Norbo to see if there’s something I’m missing here, as I’d prefer to use the much more flexible BEPU if I can.

If I had to take sides right this moment though, then I’d say that it looks like I’m stuck with my own physics simulator.

Edit: BEPU may have won. I might be able to get a slight performance improvement over BEPU with a lot of effort, but BEPU’s performance is excellent under the new test conditions, and has way more functionality than I have implemented. The only remaining question mark is the large difference in speed between different test conditions in BEPU (placement of spheres and terrain shape), where in my simulation, things are pretty a constant 30ms per update (on XBOX) across the board for 400 spheres with the same tests used with BEPU.

Anyways, terrain-sphere collision works now, and the new terrain engine is integrated into the game. Simulating 400 spheres on the PC takes only ~3ms per update (single threaded). This should make 400 spheres viable on the XBOX 360, which will have about 33ms per update on a CPU core dedicated to physics. This is about a 5-10x speed increase over the performace I was seeing with the default BEPU physics setup.

Sphere-terrain collision.