::GTapex.com:: Realtime Fakery 101: AO texture/vertex pre-lighting & Psudeo self shadowing


AO and  Shadow Fakery
Here i'm going to go over some thing which is pretty important, which is GI/AO shading on the bodywork and parts of the car.

To a certain extent this is kind of a last gen worry, with the advent of SSAO which does a fairly reasonable job at giving the onscreen models more visual weight..... but the reality in my opinion is that  a) it's pretty hardware dependent b)engine dependent c)still not good enough that i'd personally give up pre-lighting my cars (baking, if you will)

For the bodywork, the easiest and most common method is just to bake the texture, so its got a nice bit of shading to it. In the past like on the AE86 Trueno model and Mustang i spent ages drawing the shading by hand to try and emphasise the shape of the car. Obviously its a lot easier now since you can just render a texture straight out of your 3D software with the bake, or even bake the lighting data into the verts themselves (though for this to work well, the model reall does need a decent number of verts.

So the two main techniques i'll cover is
a) Pre-lighting using Rednder To Texture (Baked Texture)
b) Pre-lighting using Vertex Paint (Baking stored in vertex channel)

Most of the time, texture baking is the easiest and most portable solution since almost any engine will support a texture. Vertex channel information, while many games support it, alot don't, fortunately for everyone the Xoliulshader 2.0 supports vertex color.
Why Bother?
In this first set of images from racer there car just uses the paint materials etc, but with flat diffuse colors, ie the paint work is just white, there's now shadowing, or even a shadow plane underneath the car (not necessary in some games, as they'd probably have a cast shadow/ground plane of some sort.
Not only is the paint very flat looking, with only subtle hightlights being picked up on a few extremeties, the brake rotors are overly brigght, and the tyres just hang underneath and are evenly lit (tyres are one part that can't be just textured with shadowing since they rotate)
On the other hand in this example, the base texture has been shaded, its still white on the top, but the sides have been shaded to give the car a bit more substance. this of course needs a full body unrwap which is why more often than not most people just hope for the best and leave the car a solid color. But when directly comparing a pre-lite car and a flat textured car, the benefits are immediately noticable.
To be fair the comparison is bit extreme, since i removed the shadow plane from the first image, (it's actually a rendered plane of the shadow cast by the car, to keep the dimensions and shape etc. correct). but i believe the subtler differnences are also noticable.
The model has a self-shadow model attached to the underside to provide shading for th wheels, brake rotors and calipers, giving the wheel area a more solid and realistic feel.

Admitedly the one major problem with static shadowing, is what happens when the car is upside down, surely the shading is wrong, yes this is true, but the number of times this may happen is offset by the time it is the correct way round.
Example image break down
The above example image consist of two parts, the static body texture or bake which gives model a more realistic feel

This is the easy part as the bodywork of the vehicle is static so they shading can be either applied by a full body texture, or if support this information can be held in the vertex channels.
The second part is a weird one, i think its a good idea..... most other people don't and thats a self shadow model.

The shadow models consist of two parts, the first part is the ground plane which is very handy when it comes to viewport renders, and/or the few games that support a per car ground plane.

the second is the wheel shadowing model.
The reason the wheel needs an overlayed model is simple, unlike the other parts of a car they arent static, so it's not possible to draw the shading into the texture, or bake it into the vertex information. So to get round this the model emulates the areas in which the shadows would be cast by the bodywork.

Its farily self explanitory as a technique.
Texture or Vertex Prelighting?
Below are some example images of the same car, one is pre-lit using a texture bake, the other has the pre-lighting saved in the vertex color channel, aside from the obious differnence in lighting due to the original body texture being rendered out a while back, so the settings are different for the most part there is little difference in the look of the bake.

Noticeable differences.
Render to texture is limited to texture size, currently the model uses a 2048 texture, and there's a slight amount of visible grain since the samples werent set high enough in the rendering phase. On the other hand that because the shading is over the verts you get a smooth gradient from one vert to the next without the grain etc of a rendered texture, and you're don't need a texture at all to acheive the same look.

However haveing a texture can allow for additional shading etc, that might not be in the geometry e.g the embossed text on the rear bumper is added later in photoshop, and lacking from the pre-lit model. 

Vertex pre-lighting depends very much on the presence of verts, so there are areas where there is a lack of detail, or artifacts caused by overlayed geometry.
   - The bootlid has strange shading due to the overlayed badge
   - Snapped up polys, and un stripped tris can cause some slight artifacts.
Of course these can all but subtley tweaked later on in vertex paint, but detailed self shadowing of overlayed geometry generally works better when the pre-lighting is texture based.

In the end, upsided and downsides however you choose to pre-light your model, ultimately it depends on if your engine supports vertex color etc, which look your prefer, or if your just too lazy to unwrap a model. A combination of the two can work equally well, with any finer details being added in through texture,
Prelit using Render to TexturePrelit using Radiosity assigned to vertex color
Render to Texture Pre-Lighting
This part is all about how to "bake your texture" using the Render to Texture funtion that max as, essential what this does is render the shadowing onto a texture for you assuming your model has been UWVmapped. It's also a pretty handy techique for creating textures, say if you modelled out a high poly model to create a normal map for use on a lower poly texture.
let's get baking..
1.start by making your car a default material..(and make the materials diffuse color white)
2.the light creation menu and select skylight (make this white too) the place it in the scene (it doesnt matter where)
couple of other things you need
1. an omni placed above the car,
2. and inverted box for shadow (depending on the size of the box.. you're shadowing will change so make the box tighter around the model and the shadowing will be more intense,, make it wider, andyou'll get an overal more subtle effect)
go to render > advanced light, and select light tracer..

this will bring up the Render menu
up pops this little menu, make sure you set the
rays/sample from its default 240 to a higher number... the higher, the
nicer the baking (and the longer it will take, but its worth it... here
i've put in 2000 :O)
again, go to the render menu, the select "render to texture" this option is below the advanced lighting option you selected in the previous step.

1. Make sure you set a nice level of padding.. 2 or 3 pixels is good, this is the space around the mapping
that will be rendered, and stops any texture seems

2. Make sure, you have USE EXISTING CHANNEL ticker, otherwise it will attempt to remap your car, and this is usually a bad thing.... (and messy)

3. Click on the add button and select complete map

4. Choose your map size 1024 is usually sufficient, but 2048 is a good base for a larger master texture (takes longer though)
again, go to the render menu, the select "render to texture" this option is below the advanced lighting option you selected in the previous step.

5. Hit render and wait....
The Rendered texture will look like this.
As you can see the subtle details are emphasied by the hightlights and shadows
Now that' you've got your base layer with all the pre-lighting done, you can then texture your car to your hearts content.
Vertex Channel pre-lighting
Racer has had vetex color support for a long time, it's just rarely implmented in tracks, and almost never used on cars. So here's a rough how to on getting the same "baked" effect that can be gather from a texture, but without the memory overhead of running a large texture.

To the left is a viewport grab of the car, with just a standard white material applied (no textures etc.).
.... and here we have the exact same car, with the same material, except the  baking information has been stored in the vertex color channel.

The result isnt much different from the the baked output above (apart from maybe being a little lighter)

1. Lower memory overhead since there's less need for large textures
2. No UVWmapping required, and on race cars only the texured bits need be mapped
3. Sell static than textured shading, and Can be effected by ingame lighting (engine dependant, of course)
4. Quicker.... to an extent

1. Is vertex based, so for a really nice bake you really do need the verts (i read more detail=more polys)
2. Can lake some of the more subtle detail that a texture would give.
3. Needs support from the graphics engine.
4. Not as forgiving as UVWmapping to geometry changes.
First things first,
1. Most importantly the car/part that is going to be baked needs a standard material applied (with a pure white diffuse) remember any color int he diffuse will effect the "bake" and you'll end up with weirdly shaded, and/or colored bits
2. Its a good idea to double check that the right vertex channel will be shown, so right click and select Object Properties
-ensure the vertex channel display box is shown, or you won't see any results on screen
-since we are applying the baking to the vertex color channel, make sure that its selected in the drop down box.
Next thing to is to make the vertex color information visible, so we will go to the display options menu, and tick the vertex colors box in the Display properties rollout.
Assuming no vertex colors have been applied your model will go bright white (since it's showing the raw color of the vehicle and ignoring the default lighting)

It is possible your car could go weird and look fragmented with crazy tri's etc, that'll be down to having a previous vertex color applied and changes in the mesh, or polys being split etc.
Next bit is very similar to the previous section on baking using render to texture.
You'll need to put a light in the seen, as previously, though the extra omin for directional light isnt really necessary,

Model setup before pre-lighting

In the example you can see i've hidden the optional bonnet lip, and other mod parts for the car, however i've left the spoiler purely to show the self-shadowing that is obtained using vertex shading.

1. hide overlapping/optional geometry, while its not allways necessary in the case where model doesnt have any optional extras, a spoiler for example will cast shadows onto the bootlid, so if you have this as a removable model, then when it is not present a dark shadow will end up cast on the bootlid, which will look weird. One way around this is have the entire boot as a swapable object (you'll just need to remeber to flag it in the .ini as a generic model or you'd be running about with no bootlid

2. Wheel are always best lit seperately, and rotated 90 degrees so they are facing upwards, otherwise they will have strange directional shading.

Now that the model part is done, you're read to move to the pre-lighting part.

Vertex Paint & Pre-Lighting
The image might be a bit confusing because there's alot of menus on it but it'll all make sense, with an explanation of the work flow.

1. first you want to apply a vertex paint modifier, a dialogue box with the paint options will pop up, as well as the menus for the vertex paint.

2. You can do one of two things now, either ensure the "vertex color" radio button is ticked in the Paramaters box, or you can add an additional layer (from the layer section of the vertex paint menu) this will bring up another box where you again choose from vertex color/alpha/illumination etc.
The only advantage an extra vertex paint layer gives is you can, change the opacity of it if the shaing is too strong, other than that...

3. You'll then want to bring up the Radiostiy dialogue box, this is found in the Rendering menu at the top, (Rendering>Radiostity),

4. Now, if you've not set up the radiosity before, a box will pop up about exposure control, since that is something you'll want to use, to ensure the shading is nice.

Exposure Control
Choose Logarithmic Exposure Control, ensure the Exerior daylight box is ticked at the bottom (you're bake will be overly bright without it)
You can tweak the brightness contrast till its a nice level.

Hit the render preview, for a quick preview, and then tweak some more until the shaidng levels are about right.

Advanced Lighting box
Now you can go back to the Advanced lighting box to set up the final bits for the radiosity settings,

1. We want it to be as accurate as can be so, change the defualt 85% quality value up to 95%+

2. Indirect lighting should be set around 3, this is the light that is bounced etc, you'll find that when you change the value up it blurrs the shading more over the model, so if you find areas of the model are too sharp this is where to change it.

Then Click on the start button. the computer will whirr away and you'll a "baked" looking model on the screen.

If this is to your liking, then we can move on, if its not its pretty easy to tweak any exposure control / or radioistity settings since the result will show up on the model.
BUT do not change anything in the seen ie hiding/unhiding parts etc. because the radiosity solution will become invalid.... and you'll have to re-run the radiosity process.

Applying Radiosity to Vertex Paint
If were happy with the Radiosity solution we can now apply this to the vehicle and have the data stored in the vertex color channel, and this is the easiest part of the process

1. Select the models in the scene if they havent already been selected.

2. You should still have the vertex paint modifier, so go the "Assing Vertex Colors" part of the rollout, here is where the magic happens
- Light model should be set to lighting+diffuse
- Rendering options should be set to, Radiosity, Re-use Direct Illumination from Solution

Hit Assign.....

Again max will go away and do its thing, once its done collapse back to editble mesh/poly and you're done.