Flexible&fast Shader Engine (ENBM)

Community Forums/Showcase/Flexible&fast Shader Engine (ENBM)

bytecode77(Posted 2008) [#1]
hi!

this is a shader engine, i made. yet, it supports ENBM and refraction. when i have time, i will implement dot3 and specular support.
you can do apply reflection using one line of code, then another line to ENBM the entity. you can do refraction as well. this can be used for realistic fire!

you may use this where ever you want. just say it's from devils child ;)

note: it is not compatible with the devil shadow system. if you want to use shader AND the dss, please use the build in shaders in the dss.

screenies:




download: http://stuff.dev-ch.com/index.php?article=stuff_shaderengine (3.83 MB)
bye!

Last edited 2012


AJirenius(Posted 2008) [#2]
Something seems to be wrong with www.dev-ch.de. No link does work. I would really like to try this out!


Dreamora(Posted 2008) [#3]
Looks great and runs very good (especially given the fact it uses tweening mechanisms)

I especially like that the refraction even works when you are within the sphere. This finally allows us to create real underwater games :-)

280FPS and above (C2D E6600 @ 2x3Ghz, 8800GTS SC, 2GB RAM)


bytecode77(Posted 2008) [#4]
i just downloaded it, 35 secounds after your post and it works for me... do you see any screenshots?
anyway, call http://dev-ch.com/ and tell me if you see the portal.

Last edited 2012


bytecode77(Posted 2008) [#5]
yea dreamora. you can even look trough the sphere and see ENBM on the other side!


here you see the scene and below on the pic you see the ENBM bumpmap overlay.
so i set the alpha of some of the meshes to e.g 0.5 and you can see trough them!


Pete Carter(Posted 2008) [#6]
dead links


AJirenius(Posted 2008) [#7]
Cannot see screens, links does not work and http://www.dev-ch.de/ doesn't work. Somethings wrong


Lui(Posted 2008) [#8]
For me the link works fine...

is looks very impressive.
Most of all i like the effekt on the cube :)

I got between 250 and 300 fps (Ati HD 2900 XT)

is this campatible with your particle-system?(can the effect also be used in the particles from the system)

(damn why did i buy the ashdow lib for this? ^^)


boomboom(Posted 2008) [#9]
Down for me


bytecode77(Posted 2008) [#10]
* i have 200-300 fps, too. (ati x1900gt)
* i like the fire effect most :)
* a tiny hack could make it compatible with the Devil particle System...

i can not explian the dead links. could you guys be more specific please?
what does that mean? 404? 403? or no server found?
did ya check your firewalls?


AJirenius(Posted 2008) [#11]
Now everything works again, no explanation either. Unstable?

Well, back to the content, DevilsChild: you have just right now helped me choose my next platform for my next game. If I'm able to use this engine I will surely use Blitz3D for my next game as well because this is EXACTLY what I needed.

Awesome!


bytecode77(Posted 2008) [#12]
cool :)
but dont forgett to set every other mesh as a NEUTRAL mesh.

ps: when i have time tomorow i will try myself on dot3 mapping and specular dot3 mapping!


Pete Carter(Posted 2008) [#13]
yeah works fine now, downloading looks cool. keep it up devils child whats next?


bytecode77(Posted 2008) [#14]
maybee dot3 mapping in this engine :)


t3K|Mac(Posted 2008) [#15]
this looks really great!!


degac(Posted 2008) [#16]
Great!!!


mongia2(Posted 2008) [#17]
good work!!


Ross C(Posted 2008) [#18]
I see your using normal maps. Do they work properly in your engine? I mean, can you change the light source and use tangent space maps?


bytecode77(Posted 2008) [#19]
tangent space maps - what's that?
anyway, the normals maps are rendered to the normal-map-overlay i posted above. thats the one screenshot which is entirely purple.


Dreamora(Posted 2008) [#20]
what he meant is if they are a dynamic effect like Dot3 bumpmapping that reacts to light ... which he most likely would know: no as you've posted above that you might integrate it as well ^^


bytecode77(Posted 2008) [#21]
if i integrate it, it could be adaptive to every light source, or it would be at least adaptable to a lighting box, which ever would be my les satisfying variant.


boomboom(Posted 2008) [#22]
wow, its really great that somones basiaclly put together all the more advnaced techiques that have been knocking around blitz in demos for a few years now, into one library. I havn't downlaoded it yet, but is this pure blitz or a dll as well? Will your dot3 mapping update based on light locations too?


bytecode77(Posted 2008) [#23]
i dont know if i will support dot3 or dot3 using light sources. i will play around with in:)

and yes, you have to use two dlls. i included them into the download. in fact these are the exact dlls you get by downloading the devil shadow system.


Ross C(Posted 2008) [#24]
Hehe, caught out again not reading the whole thread :o) Would be awesome if the dot3's reacted to the lights ^_^ Just what i need!


puki(Posted 2008) [#25]
Oooh - this is all very sexual. Looking at the entire scene I was getting 480+ FPS. However, a bit of a stuttering when moving the cam with the keys (something not right there).


puki(Posted 2008) [#26]
Do you need to load textures and sprites in the main loop?

I certainly hope this is optimised as far as possible as I really won't stand for any kind of sloppiness.


bytecode77(Posted 2008) [#27]
yes, it is optimized by blitz3d. i'm not loading new textures every frame. besides, this example prints the used vid mem on the screen, so see for ya self ;)


puki(Posted 2008) [#28]
What's with the stuttering?


Beaker(Posted 2008) [#29]
Looks really good. Only one oddity is that the reflective sphere looks like it is reflecting rainbox colors on my computer, not like your picture above. [EDIT: looks fine when the camera is inside the sphere tho!]


Naughty Alien(Posted 2008) [#30]
>>when i have time tomorow i will try myself on dot3 mapping and specular dot3 mapping! <<

I'll kiss ya for that ! :)


Dreamora(Posted 2008) [#31]
Yeah great, he really considered to do it and noooowwww?
Thanks Alien *hehe*


Mustang(Posted 2008) [#32]
Cool!


Vorderman(Posted 2008) [#33]
re. the slightly stuttery camera movement - I just tried it with a local test program, and I can get the exact same result without the shader system in use by using just "RENDERWORLD" rather than "RENDERWORLD frametween", ie. not using the tween param of renderworld.

It appears that ShaRender() is not using the same method of tweeing?


bytecode77(Posted 2008) [#34]
sharender() is not using frame tweening. thats only the example. in your games, you can use whatever you want. of course you can modify the code if it doesnt fit your requirements yet.


bytecode77(Posted 2008) [#35]
ok. new update
download link is within my first post

this gives a little preview of dot3 mapping. it does not support dynamic light sources, but a light cube.
thats all i can do yet (sry)

hope you like it :)
bye!


boomboom(Posted 2008) [#36]
Very cool. Are you going to sell this or will it be free? Also, what is going on in the dll's Is it just maths or directx calls too?

I am just a little cautious about using dlls for future compatibility etc :)


boomboom(Posted 2008) [#37]
Hey, I did notice 2 things:

1) when a refraction happens over the edges of the rendered world, there is black around the edges. For example, go inside the refraction cube. I guess this is a limitation of blitz bot rendering things outside of the screen. But maybe the option to stretch hte last pixel so it doesn't seem as noticeable?

2) If you go inside an object, for example, placing your camera inside a column or the normal map sphere it will remove all the refracted effects.


bytecode77(Posted 2008) [#38]
it will not be commercial. you may use it wherever you want, and you can change parts of the code.
but please give me a little credits :)

1) thats normal, too. if you weaken the enbm using shaSetENBMInfo(), you can avoid this problem a little. but you can not stretch the border pixels.

2) true, but thats because backface culling is disabled. thats normal, and is this really a problem? ;)

thats how the screen overlay works:



Doggie(Posted 2008) [#39]
your ice cube is the coolest thing I've seen in Blitz yet!


bytecode77(Posted 2008) [#40]
thx:)
wow, you guys really like my icecube ;)
i thought the fire would attract you most.


El Gigante de Yeso(Posted 2008) [#41]
Awesome work Devils Child. I'd love to find out more about how EMBM works, have you used any reference article or source? I'd be interested in knowning if there's a way to simplify the rendering process to get simple refraction (for use in water). Anyway, good job, I'm loving it and trying to find out how it works =D


bytecode77(Posted 2008) [#42]
it works simple. all in need to do is to give the bumpmap and the texture i wish to stretch to the dll (by tom ;)) and then it works pretty for it self. instead of applying bumpmaps, i render the bumpmap scene and copy it to the screen overlay texture (press SPACE to see it).
i'm using render-to-texture here.


AJirenius(Posted 2008) [#43]
Yes, I'm scratching my head with water refraction as well. I want to use your shader for water but I just cannot get to see how


bytecode77(Posted 2008) [#44]
well it is very easy in this system. just look how refraction is done on that sphere and use a water plane mesh instead of a sphere...


AJirenius(Posted 2008) [#45]
yes, but somehow the water must animate as well.


bytecode77(Posted 2008) [#46]
you just apply the command shaSetENBMMesh() for every frame. if you apply that command twice on a mesh, then you just change it.


bytecode77(Posted 2008) [#47]
ok. here is the new update.

new is:
* i brought in a little speed by optimizing the render pass
* i put in a water plane to show how to do water. it worked with the other versions, too. the water slows down the whole app a bit, so dont worry.

all versions are getting all slower because the example contains more and more little examples:)

download the latest update from the first post


chwaga(Posted 2008) [#48]
devil, did i ever tell you I love you?

but i still don't get the name, it's kinda repelling...


















Except, why is it that a sphere that I make with just reflection (no enbm) is interfered with by the behind refractions, AND NOT THE SPHERE YOU MADE (that reflects [Obj1])


John Blackledge(Posted 2008) [#49]
Hmm. There does seem to be a slight problem, even with the latest Nvidia drivers.
(Spec: Pentium4 1.60 Ghz 768MB Ram, Nvidia GeForce4 Ti4200, DirectX 9)
I guess you need a newer card for shaders?



bytecode77(Posted 2008) [#50]
what the? did this happen with the older releases, too?

ps: try these resolutions:
* 800x600
* 512x512





Except, why is it that a sphere that I make with just reflection (no enbm) is interfered with by the behind refractions, AND NOT THE SPHERE YOU MADE (that reflects [Obj1])

because you have to do this:
shaSetNeutralMesh(your_sphere) ;A neutral mesh

this lets the engine know that it is no enbm or dot3 affected mesh.


nrasool(Posted 2008) [#51]
Congrats Devils Child, well done, and thanks for providing this to the community. Much respect to you :)


John Blackledge(Posted 2008) [#52]
Older versions? Yes, same with both.
I just ran the Sample.exe.
I don't believe there was an opportunity to change the res.
I immediately downloaded and installed the latest Nvidia drivers and rebooted, and reran Sample.exe.
No change.


puki(Posted 2008) [#53]
Works fine for me - apart from the continual bent stuttering that seems to be down to me to fix for myself (considering it is still there) - I have made a note of this and will keep it on record for 6 months.


Beaker(Posted 2008) [#54]
Eradicated the rainbow colors by changing the texture flags from 385 to 128+1 in function shaSetReflectionMesh() in ShaderEngine.bb file.


puki(Posted 2008) [#55]
Eh? I didn't get rainbow colours. I want rainbow colours.

Add rainbow colours in or I'll snap your little maggot legs.


Naughty Alien(Posted 2008) [#56]
..why Dor3, no matter I turn on/off light or reposition it, still getting lightened from same position?? What to do with that cubemap lightnormals in order to make it responsive for light position?


El Gigante de Yeso(Posted 2008) [#57]
Thank you Devils Child. I kinda figured it out how it works. Anyway, I'd like to point out that multitextured objects tend to distort the normal overlay, so I made a workaround that sets the other texture layers other than the actual bump to shaBumpColorTex, so they do not interfere at all.

I'll make sure to check this out deeper and find out if I can have it working a different way or something (moving the copy objects to the infinite plane kinda seems too much a workaround :s)


bytecode77(Posted 2008) [#58]
*dot3 still uses lightboxes which doesnt depend on light sources at all.
* what do you mean by rainbow colors? i wanna see that :)


Naughty Alien(Posted 2008) [#59]
>>*dot3 still uses lightboxes which doesnt depend on light sources at all.<<

thats fine but is there way to manage lightboxes in order to get dot3 effect movable??


bytecode77(Posted 2008) [#60]
no, but there are other ways to get dot3 working clean. i will check out these.


Naughty Alien(Posted 2008) [#61]
gooood..thats my mann ;)


bytecode77(Posted 2008) [#62]
i still am looking for the example with the colored light method for making dot3 effects. because instead of colored lights, we could use colored cubemaps!


Naughty Alien(Posted 2008) [#63]
is that other ways of doing dot3 (movable one) going to work over animated characters too??


bytecode77(Posted 2008) [#64]
dont know, but this one with the colored cubemaps i figured out works similar to the one with colored lights.
if i only could find the example, it would help me a lot...

at first i will test it with colored lights, but i dont know how to position those lights...
i'll just try...

edit: just found :)


Sledge(Posted 2008) [#65]
This really is lovely work! The only effect that seems to give my Intel(!) chipset any trouble is the reflective one and it's always had trouble with cubemaps anyway...




bytecode77(Posted 2008) [#66]
hm weird. looks like it is not rendering a cubemap face.
and you know, i dont apply the bump maps on any cubemaps!


Beaker(Posted 2008) [#67]
Sledge - try my fix for rainbow colors in the cubemaps (above).


Yo! Wazzup?(Posted 2008) [#68]
Red Oktober from SyntaxBomb said it's "good stuff" :D


bytecode77(Posted 2008) [#69]
cool!
show me the link!
eidt: http://www.syntaxbomb.com/forum/index.php/topic,1074.msg9732.html#msg9732


Mortiis(Posted 2008) [#70]
It's beatiful. I would love to see it expanded. Maybe some fullscreen effects like glow? dof? I don't want to buy aShadow.

Great work as always!


bytecode77(Posted 2008) [#71]
you mean post processing? i was thinking about that.
i'll look at it. but that doesnt mean that it will be implemented into the next release!


edit:
from the later posts John Blackedge
I don't believe there was an opportunity to change the res.
in the line "Graphics3D....." change the screen resolutions and look if something else happens!


Ross C(Posted 2008) [#72]
Devil. Cubemaps work for DOT3 maps. You can change the lightsource, by writing color data directly to the cubemap. Because cubemaps works with the normals of a mesh, then it auto updates so the DOT3 responds to changes in the mesh.

Only problem is change the light source. As i said you need to shift the cubemap colour data on the texture. Which can be slow. I've seen people use a very low res cubemap before. You still have the speed hit of writing to a texture to change the light direction though.


bytecode77(Posted 2008) [#73]
hm. and if i change the cubemap in real time, i would have to change all cubemaps for every mesh seperately! that would be slow like hell. also i would only be able to use one light source.

if you guys find a code archive entry that is really usable, feel free to post it here :)


Ross C(Posted 2008) [#74]
Well, all the meshes should be using the same lighting cubemap. So one update will affect all the meshes using it.


Ross C(Posted 2008) [#75]
Devil, if you run this code, it will generate and save a cubemap in the directly it's run. If you apply this OVER a normap blend and set the right blend mode (can't remember), it will give you a light.




bytecode77(Posted 2008) [#76]
that generates a cubemap that is very similar to my lightnormals.
but does that change the whole thing?
sorry, if i didnt understand what you mean!

thanks for the code anyway!!


Ross C(Posted 2008) [#77]
Ok, check this out:

http://www.rosscrooks.pwp.blueyonder.co.uk/normalmapping1.zip

It has the normal map applied, PLUS the color cubemap. Press key 1 to 9 to animate. Notice how the normal updates as the model moves. The other map, lmap, is simply to add some colour.

The idea comes from a user called MWS? MSW? one of them :o) so credit to him for the techniques.

To change the light source, you would need to physically write to the cubemap and change the way it's laid out. But you can get away with using a smaller cubemap.

EDIT, i changed the the actual dragons texture, so you can actually see the difference.


bytecode77(Posted 2008) [#78]
thanks, i will check this out right now:)
edit: sure! my code works on animated meshes or MD2, too. but this zip file you posted still doesnt care about the light position, just like my engine!


bytecode77(Posted 2008) [#79]
ok. rewriting the cubemap in a small res would be possible.
that would mean that i have to look at all lights that i have and then write for every mesh a cubemap. i just dont have an idea how to do that!


Ross C(Posted 2008) [#80]
Yeah i know, BUT, if you physically alter the cubemap, you can change the light direction. I managed to get the cubemap down to 36x6, so it shouldn't take very long at all to move the light source :o)


bytecode77(Posted 2008) [#81]
sure it is fast. but i need to know how to generate these cubemaps. important is the position of the light source relative to the position of the object. then, how do you render the cubemaps?


Ross C(Posted 2008) [#82]
Well, i'm going to give that a go. Just a case of shifting the pixels around.

Give this a whirl:

http://www.rosscrooks.pwp.blueyonder.co.uk/normalmappingnew.zip

Press the space bar. You will see the source of the light changing once.

I'm not sure how small you can get away with making the cubemap. Surely there must be a shader you can use?


bytecode77(Posted 2008) [#83]
i'll look at that, too.
btw: i have a slight idea making this in real time! i'll try it now!


Ross C(Posted 2008) [#84]
Oh? Care to share :o)


Ross C(Posted 2008) [#85]
Oh, i have an idea also. You texture 6 cubes with that cubemap. Position them so if you snapshot the screen you get a view of left,right,top,bottom,front and back. Then you simply copyrect to the cubemap texture. I'm going to try that out.

You would just rotate the cube to the light rotation and it should work. And shouldn't be too much of a drag on the computer either.


bytecode77(Posted 2008) [#86]
you mean a screen overlay?


Ross C(Posted 2008) [#87]
Imagine, for simplicity sake, having a cube. You texture it so all the sides match the cubemap texture. Now, in order to change the light source, you have to manipulate the cubemap. The cube is a representation of the cubemap. Therefore by rotating this cube, you are essentially rotating the light source. But you need to translate this into a cubemap texture. So, you need to take shots of the cube from the +Z,-Z,+X,-X,+Y and -Y. Creating the new cubemap.


bytecode77(Posted 2008) [#88]
i think what you want to explain here was actually my idea!!!
and i'm half way trough it. i think i got it soon!


Ross C(Posted 2008) [#89]
Race you :oP


bytecode77(Posted 2008) [#90]
ok. here is the new update!
it supports light movement. but only of one light source yet!

download from the first post or here:
==> http://projects.dev-ch.com/article-shader-engine

Last edited 2012


Pete Carter(Posted 2008) [#91]
even better well done. one thing you could make the light not completely bleech out objects its a bit to white at the brightest point, i think it would look more natuare abit less extreme. but very good work


Ross C(Posted 2008) [#92]
Dam you beat me :o( I'll check it out though :o)


Ross C(Posted 2008) [#93]
Very impressive man :o)


Ross C(Posted 2008) [#94]
I have had a thought about multiple light sources. create two cubemaps, then overlay them, alpha them 50-50, or 33-33-33 for three lights. The combination, using the right blend mode, should do the trick. Only for directional lights though...


Ross C(Posted 2008) [#95]
Forget this... Forgot you can only copy to one face at a time. Pity you couldn't change the texture flags during runtime :o)


Ross C(Posted 2008) [#96]
Say... you have tom's dx thingy yeah? Can't you render to texture with it? Just render the the screen straight to the cubemap texture buffer.


bytecode77(Posted 2008) [#97]
umh. the method i'm using is very slow. i was thinking of making this with vertex colors. if i knew how, i could do it.

it would look exaclty like the mesh with the cubemap, but it doesnt need any rendering!

ps: please dont post 5 times in a row. i've never seen anyone posing more than 3 times in a row. you really broke the record! ;)


chwaga(Posted 2008) [#98]
WHERE IS TOM'S THING?!?!?


Ross C(Posted 2008) [#99]
Heh, never realised :o) You can alter the vertex colours on the model and that changes the light direction. Only problem is, this is pretty slow too, and you need to update every mesh individually.

Can't you use the render to texture from tom's dll?


bytecode77(Posted 2008) [#100]
it will be faster than the old one anyway.
and yes, i am using render to texture from toms dll. but i'm not using render to cubemap, because i dont know how to. there was no demo source yet!
WHERE IS TOM'S THING?!?!?
what do you mean? does the update not work or are you looking for toms original demo?


Ross C(Posted 2008) [#101]
Hmm, i thought it would be the same thing to render to a cubemaptexture.


Chroma(Posted 2008) [#102]
That heat shimmer is very very cool. Me thinks I'm gonna add this stuff to my F-16 flight sim.


Naughty Alien(Posted 2008) [#103]
..boy, this is nice so far..if ya keep going I'll donate ya :)


bytecode77(Posted 2008) [#104]
..boy, this is nice so far..if ya keep going I'll donate ya :)
thanks dude;) also i'm not very good in taking money from people ;)
i'ts ok. i see you apprecheate my work :)

btw: there is one line you may change. in the shaderengine.bb is a line where a texture with the dimensions 64x64 is created. change these to 8x8 instead.


chwaga(Posted 2008) [#105]
i mean, where can i find tom's dll??


bytecode77(Posted 2008) [#106]
its within that directory. it just has an other name...


Sledge(Posted 2008) [#107]
Sledge - try my fix for rainbow colors in the cubemaps (above).

No joy I'm afraid!


puki(Posted 2008) [#108]
What am I supposed to notice different about version 7?

EDIT:
Ah, I see - it has a moving light.


bytecode77(Posted 2008) [#109]
i made a test concidering vertex-color-bumpmapping.
i also implemented it, but it is quite buggy. it doesnt look satisfying and doesnt bumpmap at all if the light is "behind" the mesh.
it is a little slower when using only one dot3 mesh, but it is significantly faster if using several dot3 meshes!!




Ross C(Posted 2008) [#110]
It surely will be slower using multiple meshes and only one light though?


bytecode77(Posted 2008) [#111]
but compared to my old cubemap method, vertex colors are faster using multiple meshes!


Shodan(Posted 2008) [#112]
Very cool stuff Devil!

One Question:
How i use the Shadow Functions together with the new stuff? Thats where peak of all.

For your problem with light-directions and dot3-mapping:
Several month ago iŽve posted the BSM_Renderer in the Toolbox/other Libs. It uses Cubemaps to manage "dot3-lightning". The Cubemaps are managed as fast as possible:
-created Cubemap could be given to other objects, when light-direction is the same for all ( i.e far sun)
-angle of light-direction will be checked, so an cubemap-update is necessary.
-amount of max. updated cubemaps per loop can be set.
..and so on

Take a look at this. Hope it brings you some ideas.
IŽll try to test it with your example. Tell you the results.

Shodan


bytecode77(Posted 2008) [#113]
hm. it is not compatible with the devil shadow system. also if i would implement it into it, it would be buggy anyway, so i decided to release it as a standalone engine!
ps: i will take a look at your stuff soon. thx!!


El Gigante de Yeso(Posted 2008) [#114]
Yo Devils Child. I've made my own little rewrite of the engine, based on your library but with a new optimization technique.



Since on your engine you need to render twice all the geometry within the scene, one for color map and another for normal map, on scenes with heavy geometry this could be a great bottleneck (also, you have the inconvenience of registering all objects, even if they are EMBM-mapped or not).

I workarounded this by playing with the ZBuffer. I've hidden first all the entities that aren't EMBM-mapped and rendered them in normal space to the color map. After this, I shown and offsetted all objects to infinite (like you did in your system), and rendered the EMBM-mapped normal-mapped entities to the normal map (without clearing the zbuffer), and finally rendered the EMBM-mapped entities (yet again, without clearing the zbuffer).

This allowed to render only the needed stuff between passes, thus gaining a good amount of perfomance :) However, not everything is good with the method. Sometimes there appear to be some errors when rendering the normal map,wich I couldn't find out why they ocurred. There's also some z-fighting due Blitz 3D not being precisse enough when rendering both the EMBM geometry and EMBM normal-mapped geometry (but doesn't matter much if you're just going to use it for refraction effects anyway).

So please, test it out, and maybe you could find a way to enhance it :)

Download link


bytecode77(Posted 2008) [#115]
cool. thanks for the download link :)
i will check this out soon :)
btw: good looking game! would like to see something 'wearing' my engines ;)


John Blackledge(Posted 2008) [#116]
Sorry D.C. - still the same effect with your new version 7 in any resolution, as per my screenshot above.

i.e. All effects offset away from the objects, and chunks of water appearing over the uprights.


jfk EO-11110(Posted 2008) [#117]
Great stuff, DC! Thanks a lot man.


Reactor(Posted 2008) [#118]
"...would like to see something 'wearing' my engines ;)"

Once things really get cranking I'll happily wear this engine down for you :)