Swift's shadow sys headache

Blitz3D Forums/Blitz3D Programming/Swift's shadow sys headache

xmlspy(Posted 2006) [#1]

Why the heck does the shadow display on the cube, but not on the level? Yes, I'm setting the level as a shadow receiver.


Mustang(Posted 2006) [#2]
Is your level normal geotmetry? I'm not sure if it works with BSPs for example.


xmlspy(Posted 2006) [#3]
I think I might have to end up making a simple four face wall into more sections so they might work properly. I noticed that if I scale up the cube it will still only receive shadows as if it was a 1x1x1 scale.


(tu) sinu(Posted 2006) [#4]
If the mesh is a anim mesh, don't you need to setup each child to recieve shadows?


sswift(Posted 2006) [#5]
Did you scale the level using ScaleEntity? Try ScaleMesh.

Also, what's the scale of your world? Ie, how big is that cube? If your objects are really big, or really small (ie, that cube's size is way more or less than 1 unit) then that could cause issues. In that case you may need to adjust the scale or try tweaking a few of the constants in the system, like these ones:

; Set this to a value greater than 0 if you wish to nudge the shadow meshes in the direction of the normal
; of the underlying surface.
;
; Nudging the shadow takes care of z fighting issues, but does take a little more processing time.  If you are
; rendering your shadows in a lighting pass before you render your world geometry, then you can probably set 
; this to 0.
;
; The amount you should nudge the shadows depends on the scale of your world.  If 1 unit = 1 meter in your game
; world, then you should set shadow_nudge to be really small!
Global Shadow_Nudge# = 0.01


; This is the amount the add-caster functions multiply the true radius of the mesh by. 
; We fudge the numbers a bit so that when we render the texture the model doesn't touch the edge.  If it did, then
; the UV clamping would create streaks across the whole shadow mesh where there is no shadow.
Global Caster_Radius_Fudge_Factor# = 1.25


; This value pushes the caster clipping plane from it's center, towards the light by some fraction of it's radius.
; If this value is 3, them it pushes the plane towards the light by a third of it's radius.
; If you get no shadows, try setting this value to 1, and then increase it if you see shadows being clipped improperly
; near the base of the shadow casting object.
;
; The caster plane defines where the shadow volume begins.  If it were in the middle of the object, then a crate sitting on the floor
; would have the shadow beneath it clipped along it's diagonal.  So I push it back towards the light a bit to fix that problem.
Global Caster_Plane_Fudge_Factor# = 2.75



Another thing to consider is that if you want the system to run at a good speed, you'll need to split your level up. Ceilings for example do not need to recieve shadows, and if the system can cull all the polygons in every room other than the one you're in with a few sphere tests because they're all seperate entities, then that will be much faster than testing each individual polyogn in the level for every shadow every frame.


Btw, the pivot point of your character might matter as well. Is it at his feet? That's a no-no.


Sorry it's so finicky, but because I couldn't demand people build things to a certain scale, and because you can't get vertex positions for animated meshes, and just for speed reasons, it had to be done this way.


xmlspy(Posted 2006) [#6]
The cube is at normal 1x1x1 units.
The animated character is shadow casting recursively.
The level mesh is being scaled down a lot, since 3D World Studio exports huge levels(I used scaleentity)
There is a pivot at the animated mesh's feet, and then the mesh is being parented to the pivot.


IPete2(Posted 2006) [#7]
I had to really rescale ss Great Britain and then tweak a few paramters but when it all began to fall into place it was wonderful, so keep tweaking.

BTW which version of the shadow system are you using?

XML Spy have you tried using another object such as a cube to cast the shadow instead of the animated character? Sometimes by trying simpler things you can uncover a simple reason for the problems you are experiencing. This brings it under your control and then you can shape it the way you need it.

IPete2.


Dreamora(Posted 2006) [#8]

Did you scale the level using ScaleEntity? Try ScaleMesh.


Reason is that the result of scaleentity is not the same as scalemesh (mesh scales the real vertex information, entity only changes the transformation matrix - ie in real vertex data, your ground floor just isn't there where it is shown due to transformation ...)


sswift(Posted 2006) [#9]
"The level mesh is being scaled down a lot, since 3D World Studio exports huge levels(I used scaleentity)"

Try scalemesh. I think I evetually made it so both work, but I forget. I may not have.


"There is a pivot at the animated mesh's feet, and then the mesh is being parented to the pivot."

Did you tell the system to cast the shadow from the pivot, or from the character? If it was the pivot, use the character instead.


You know what else it could be... It could be that you have no normals for the level mesh. I'm pretty sure there was no way for me to calculate the vertex normals at runtime, so you have to make sure those are calculated. There's a command to do that in Blitz... CalculateNormals() or something I think. I also put one in the code archives a while back that works better. It needs the normals so that it knows which direction to move each shadow vertex away from the surface so that the shadow gets drawin in front of the surface. If they're 0, it's possible that the calculation is just going NAN, cause I may have tried to normalize them because some 3D programs export them the wrong length.


xmlspy(Posted 2006) [#10]
Fixed it!
to scale the level ScaleMesh must be used
Then apply the UpdateNormal function to the level and presto! it works!.

Now I gotta fix it so that shadows are not going through walls. And that I don't cast shadows when I have a wall between the mesh and the light source.




sswift(Posted 2006) [#11]
"Now I gotta fix it so that shadows are not going through walls."


That's not really possible. At best you can lessen the problem by making sure light sources aren't so low that the character's shadow is cast high on a wall like that. And even then, shadows will still go through the floor.

The only way to fix that would be to rewrite the whole shadow casting function, sort all polygons the shadow falls on by distance from the player, then crop the shadow projection plane by projecting each polygon onto it, and then project that new projection polygon that will probably be multiple polygons and convex and stuff onto the other polygons in the level so you can draw a partial shadow on them... It would be a big mess, and probably much slower, and I certainly wouldn't want to have to try to do it!

I wonder how they fix that issue in stencil shadow systems... They don't crop all those stencil shadow volumes do they?


Sledge(Posted 2006) [#12]

I wonder how they fix that issue in stencil shadow systems... They don't crop all those stencil shadow volumes do they?



I thought they checked the z buffer of the scene('s pixels) against a z buffer projected from the light's view to determine if the pixel in question should be stenciled in or out. But, anyway, in the picture above that shadow beyond the wall should be obscured by the level's lightmap - couldn't it be blended differently and/or the lightmap re-rendered to be properly dark?


sswift(Posted 2006) [#13]
Yes, in theory it should be possible to do that, I've thought of it myself before, but I don't know for sure that it can be done in Blitz.

You would have to render the lightmaps first, then render the shadows on top of that, then render the textures on top of that.

In order to do that, you need two copies of the level. And you need to set the texture copy of the level to multiply or multiply x2 blend, while leaving the lightmap copy solid.

What should then happen is the lightmap version is drawn, which fills in the zbuffer and then the shadow and texture pass have transparency and so are compared with the zbuffer, but do not write to it. You must renderworld each pass seperately, or at least, the texture pass and the shadow pass seperately.

Anyway, there's one small problem even if the above works, and that is that his lightmaps aren't 100% dark in the shadows. So the shadows would still darken them anyway. The trick to the above is that after doing the level shadows and the texture shadows, which are all pure black, you render a quad over the scene set to add blend with the ambient light color. That brings them up from pure black, and you now have shadows which blend together nicely but aren't pure black in the final scene.

This isn't really a method everyone would want to use though to do their levels as it forces you to have an ambient light level everywhere or have pitch black shadows almost everywhere, and neither is realistic.

Hm...

If it were possible to take the lightmap and apply an inverse copy of it to the polygons of the shadow that match the polygons those shadow polygons overlay, then you could maybe blend the shadows better. The idea would be black lightmaps would produce white shadow maps, and that would in turn produce no shadows in those areas, but white shadopw maps would produce black shadows, and that would make dark shadows in those are which are well lit.

But that would require major changes to the system if it could be done, and is not something general that would work in just any old game.

Buuut... What would be easer to do, is if you could grab the lightmap brightness at each shadow vertex as it is created then you could do that inverse adjustment on the shaodw brightness. But that would be grabbing lots of pixels... Though I guess the shadow doesn't have many vertices, so it wouldn;t be too bad. But again, that's something specialized, not something I'd do, or even have time to do.


Sledge(Posted 2006) [#14]

So the shadows would still darken them anyway. The trick to the above is that after doing the level shadows and the texture shadows, which are all pure black, you render a quad over the scene set to add blend with the ambient light color. That brings them up from pure black, and you now have shadows which blend together nicely but aren't pure black in the final scene.


This would be the method I'd naturally opt for if I couldn't get a nice blend from a straightforward render. Trouble is I use an overlayed quad for reflections - if you start to stack too many effects this way then you just end up render(shadows)...render(reflections)...render(bloom)...render(final scene) and you can only show a (really nice looking) cube before the framerate nose-dives :D


xmlspy(Posted 2006) [#15]
Ok, so every example I know of shadow systems(sswift's, and Devil's) cannot stop after the first wall? What the freak. How do the gaming companies make them stop?


sswift(Posted 2006) [#16]
Xml:
Name one game (not one with stencil shadows) which uses shadows that can cast onto walls where they stop when they hit a wall. I can't think of any.



I think one or two guys there have shadows that are cast straight down. The rest don't.



All shadows cast straight down. They probably have a short range so they don't hit the floor below the catwalks. I'm pretty sure you can set the shadow fade distance in my system to mimic this.


Sledge:
I just remembered how I think they make stencil shadows stop at walls. With a stencil shadow system, every object in the level is lit by the same light sources. So if a player is blocking a light it is no different from the level geometry blocking the light in the same render pass. The stencil buffer at the end of the render just says "in shadow" or "not in shadow" for each pixel.


Unfortunately, this isn't much help for rendering such shadows with my shadow system. The lightmaps are rendered too differently from the shadows. And even if it did work, you would have to have one set of lightmaps for each individual light source.


Hm... come to think of it my idea for adjusting the shadow using the lightmap would also require one set of lightmaps for each light source.


And even if I did chop up the projection as it hit each polygon, the result would just look weird, cause now you'd have half a shadow of a guy darkening the area past the corner of a wall instead of the whole shadow, and it would look even less realistic.


Subirenihil(Posted 2006) [#17]
Look at the Devil Shadow System for shadows with higher detail in less time. Only drawback I see with the dss is that transparent polys cast the same shadows everything else does. dds does not darken for each shadow object - shadows are shadows.


sswift(Posted 2006) [#18]
Sub:
The Devil Shadow System is a stencil shadow system, and that means that it produces hard shadows, requires low poly geometry, and most important of all, it will not work with boned b3d meshes, like his character above.

Every method has its drawbacks!


Subirenihil(Posted 2006) [#19]
That's true. I'm not trying to say your method is bad. I like the colored shadows your method produces. I think I haven't looked at your newest version, I better look before I make more semi-negative comments about an excellent piece of work.

Your shadows look a lot like the ones in Age of Empires III - except they don't multicast their shadows. Can yours use the transparency of the texture to cast holes in your shadows like AoE3?

[EDIT] Wow! I just tested your new version and it is lots faster than dds (though blocky and multicasting). Great job!


sswift(Posted 2006) [#20]
"Can yours use the transparency of the texture to cast holes in your shadows like AoE3?"

Yes. Otherwise objects like trees would cast crummy shadows.

Let's see a stencil shadow system do that. :-)


Subirenihil(Posted 2006) [#21]
It is actually possible to do with stencils, but it would be very, very processor intensive - far more so than it already is - making it useless for games.

You might consider making a demo of your method that has that demonstrated, I think it would attract quite a few people.


sswift(Posted 2006) [#22]
I sell like one copy every two months now, so I don't think I'll be making any more demos for it. :-)

I think everyone's waiting for Max 3D, which I suspect will have its own shadow system. I just hope Mark chose to go for one of those zbuffer shadow methods rather than using stencil shadows, or else the shadows will be worthless for outdoor games.


angel martinez(Posted 2006) [#23]
Sswift,I have a couple of questions: Is it necessary, in your shadow system, to register every entity-mesh in a game with the system despite of that entity is not going to cast a shadow ? I ask this because it is the case with Devil's system.
And, is it possible to cast static rendered shadows from a big building without much performance cost?
Thanks


sswift(Posted 2006) [#24]
If the object does not need to cast or receive a shadow, then you don't need to tell the shadow system about it.

As for casting static shadows from a building, a static shadow is just another entity, with the polygons that the shadow hit, and a new texture. So assuming the shadow didn't end up with thousands of polygons because whatever it hit had throughsands of polygons (not what cast the shadow, that doesn't matter) then it should run just as fast as any other object in your scene.


Defoc8(Posted 2006) [#25]
To be fair..theres nothing stopping you baking shadows
for things like trees - into the terrain - as torque does &
using stencil shadows for moving objects..
Infact there are countless ways to go about implementing
shadows..if you use shaders, the doors open even further...
then again we're talking about b3d..and i dont know anything
.er..yeh..goodluck with that..now i dont
remember why i came in here :/


Tom(Posted 2006) [#26]
it will not work with boned b3d meshes


Not as fast as I'd hoped, and still some issues to resolve, but it can be done provided the poly count is low enough.


Each method has its pros & cons.

I used F.E.A.R combat as an example of LOD use earlier and I'll use it again now as it uses a mix of lightmapping, projected texture maps and stencil shadows, and looks very nice.

Sswift, you should take a look at some of my texgen code sometime. It can create projected UVs automaticaly using DX7 hardware, and it's easy enough to project a texture from any entity. How about render textures too (hardware permitting, but widely supported), they kill the need to do a copyrect.

And never forget the golden rule, 'even a blob shadow is better than NO shadows at all' :)


sswift(Posted 2006) [#27]
Defoc8:
If you use baked shadows for trees then you've just lost half the reason for using stencil shadows which is to generate realistic shadows that combine properly. Stencil shadows will not combine properly with lightmaps. Where they overlap, you'll get a dark spot. Also you've now got to code twice as much lighting code and have to wait for those slow lightmap builds.


Tom(Posted 2006) [#28]
Stencil shadows can blend with lightmaps, and I wouldn't use them for trees, I'd use lightmaps for trees.


IPete2(Posted 2006) [#29]
Not wishing to throw a cat amongst the pidgeons but what is projecotr shadows then? These are what Josh has settled for at this point with his engine, and they look fab.

IPete2.


Tom(Posted 2006) [#30]
I think he means shadow mapped spotlights.


sswift(Posted 2006) [#31]
Projected shadows are what my system uses. It renders a lightmap from the point of view of a light for each object that casts a shadow. Then it "projects" that shadow onto the geometry by creating new polygons in the shape of the gometry and setting the texture coordinates to put the projected shadow onto those polygons the right way.


Tom(Posted 2006) [#32]
What sswift said :)