Deferred Renderer
Community Forums/Showcase/Deferred Renderer
| ||
| Has been a while since I visited last here, and I have been thinking on doing a little Deferred Renderer in OGL - I have an advanced one in C++, but I was thinking "hey, I wonder how well this would go in Max..". So, I hope you dont mind but I'll be posting bits by bits implementation of a multi-pass deferred renderer here. It'll basically have the normal core things like albedo, specular, ambient shadows, multilights and shadows, bit of blur and some other stuff :) .. the only things I wish to warn people about is that this will be a NVidia only renderer.. dont bother asking about ATi because I live in Cg land.. and I dont have the time anymore to try and meet all the cross GPU issues. Feel free to do what ever you want with the code.. go crazy.. and make fun stuff with it.. is my suggestion :) First up.. simple albedo + specular + 1 light + shadows... oooooh :) Will post up the bmax code in a few hours :) This will also be a basical tutorial on how to go about making a deferred renderer engine - its actually easier than a normal pipe engine :) --------------------------------------------------------------------------------- How a general deferred renderer works: http://www.guerrilla-games.com/publications/dr_kz2_rsx_dev07.pdf Basically a deferred renderer processes the rendering of a 3D scene into different 'layers' of rendering. Most often, colour (or albedo) is one of the initial passes, where the scene is rendered without lights and only with textures. This is rendered to a texture on the GPU - NOT to the screen framebuffer. Then you also have other layers like a specular layer, an ambient shadow layer, a series of lighting layers and so on. Eventually you have a number of textures in your GPU memory that can be composited together to make a full image for the framebuffer. Compositing the images is also done through shaders, and a engine creator often ensures the ability to provide parameters to the compositors that can be modified to suit the output needed for the scene. An example would be if the scene needs to have high level speculars, then the specular compositor parameters can be modified to suit. Deferred renderers allow for a great deal of flexibility, with minor costs. Some people argue that transparent objects are an issue, but this is the case for all renderer's. Transparent objects need to be carefully scene ordered regardless of the type of engine you build. Additionally, there is discussion that Anti-Aliasing is ineffective with this method - this is true to a minor degree. Most deferred renderes have their own AA compositor passes that, usually look far better than the hardware ones in any case (see KillZone2 for example) So for our first pass, we will render a simple scene into an albedo texture buffer. And also render a specular to a texture then combine the two to see the results, and how changing shader parameters can make quite amazing results with relatively simple code. --------------------------------------------------------------------------------- |
| ||
| Sounds cool, looking toward it. |
| ||
| Yes this sounds really cool. I am looking forward to see it. |
| ||
| Wow, this sounds awesome! Also looking forward to see it. |
| ||
| Ok so where is it? ;-) |
| ||
| To be honest, I can't stop refreshing this site every 15 minutes. ;) |
| ||
| @Grover, Any news on this? |
| ||
| better if this can be mixed with minib3d |
| ||
| Soz. guys.. yes. I have a fair amount done (got called off to install a Simulator at Monash Uni for last two weeks.. doH!!!!) I had hoped to have some hols and do this!! So. To reiterate. I'll get the gear I have up over the next week (hopefully wont get called away again). I apologize for the delay.. btw the methods that Im using in OpenGL should be supported on about 90% of vid cards out there (even works on my crappy laptop :) ). Be back soon with code. |
| ||
| Sounds good, and no worries take your time. We will still be here when you get back :) |
| ||
| Yea, don't worry. We'll wait. :) |
| ||
| Oky dokes. Heres an initial section. It derives from a pixel shader section I did for OpenGL in Blitz a while back. However I revamped the code a bit so here it goes. --------------------------------------------------------------------------------- Design. I wont stress this enough, having been involved with game dev for near on 15 yrs and simulators as well, design is the number one thing you need to think of before starting any task. So, heres an overview of how the deferred renderer will be structured. 1. Generic Shader Loader/Setter 2. Simple Mesh Loader (3ds) 3. General Object/Entity Manager (with simple scene capabilities) 4. Render Loop 5. Render to Texture Passes (albedo and specular) 6. Combine Passes (with output pass) 7. Back to Render Loop --------------------------------------------------------------------------------- 1. General structured basic Pixel and Vertex Shader loading and Setting system - this is mainly for clarity more advanced systems use huge shaders that are loaded once, and jump around inside them to minimise state changes. Whenever you set a shader for use you flush the machine's state cache and prepare it for the shader you are going to use. This is a serious performance hit, so remember this is a DEMO and WILL NOT scale all that well if you intend to do substantial scenes with it. I will use ARB shaders because it will work on ALOT of video cards. Please do not whine that its not GLSL.. or some other language. Language is only a mechanism to do a job, the job I wish to do is to provide many people the capabilities to run the code. If you wish to make an ARB shader, from say HLSL or GLSL then download ASHLI http://developer.amd.com/gpu/archive/ashli/Pages/default.aspx This tool will make ARB compatible shaders that you can use directly yourself. I will not initially use any parameters (if I can help it) so as to keep the code simple and clean. Heres the first pass of it: This code is for simplistic use. In a fully fledged system you would have a proper manager to look after your materials and thus shaders (as well as textures and material data like lightmaps, shadowmaps and so on). This supports 10 shaders, theres no checking it assumes you are bright enough not to use more than 10, or set a number greater than 9 into the 'SetShader' function. Feel free to add error checking, but dont complain about it - thats not the goal here. Error checking IS however done on the shaders themselves - since you need to compile them you need to know if they are likely to work before even attempting to start your software. Heres a couple of shaders to play with. A simple vertex shader A simple fragment or pixel shader Ok, thats a start. See if you can set a shader and draw some OpenGL shapes. See what the above shader does if you use it :) Try using LoadShaders("MyVertShader.vsh", "MyFragShader.psh", 0) SetShader(0) .. Draw a cube or something... Bewarned.. it isnt overly spectacular :) ... next .. 3dsloader... |
| ||
| You could save yourself all the glGenProgramsARB = wglGetProcAddress("glGenProgramsARB"); glDeleteProgramsARB = wglGetProcAddress("glDeleteProgramsARB"); stuff by using glew.bmx that comes with bmax, then all your calls to those functions are available without need to do all the get proc address crap and make the code much easier to understand :) Look forward to seeing the next parts, hope you dont mind me changing the code. |
| ||
| Interesting stuff guys, thanks. |
| ||
| Nope. Please do NOT use glew. This is a VERY bad suggestion. If you want to write a glew tutorial go for it, but this is a tutorial with MINIMUM dependancies. GLEW is a particularly annoying dependancy when a simple one liner will do - this is a caveat I use for most of my work. If it introduces 'other' API dependancies steer clear of it. The function declarations are _extremely_ simple. If they weren't I wouldn't use it. And if you don't understand them, then maybe a little bit of reading will help (Id be surprised if anyone had problems understanding extern declarations - have been doing it in VB for 10 yrs... its NOT a bad practice, its how you make best reuse of code, without writing a dll interface!!). Also this is how the loader will work since there is no point having a tun of code when you can use a dll to help out. So again, I would not recommend the change, this tutorial is about simplicity and clarity. Nothing worse than having an example depends on twenty other things just to run it. :) |
| ||
| I dont get what you mean by glew.bmx being bad, how can using a simple piece of code that provides all the gl constants and access to all the functions added to opengl up to ver 2.0 pre defined and ready to use be bad, surely this is best reuse of code since I am not reinventing the wheel each time? Plus as far as I am aware glew is cross platform so all blitzmax users can benifit from your tutorial code. But I wont interfere with code changes again so as to not derail your thread from its purpose as a tutorial. Cheers Jon |
| ||
| Wow, cool, thanks for sharing! I've been messing around with glslang and BMax lately, so I'm looking forward to the next step of this tutorial. :) |
| ||
| Oky dokes. Step two, a nice simple 3DS Loader :) This is basically a rip of a very good piece of 3ds loader code written a number of years ago by Matthew Fairfax. http://assortedwidgets.googlecode.com/svn/ This is one of about 4 open sourced 3ds loaders you can find on the net. Its very easy to modify, and I'll have a link to the full source code so you can compile your own DLL's add animation, etc to the 3DS loader itself. A quick point about glew. glew, glut and friends are ADDON API's. They are DISTINCT from the OpenGL standard and remain so. If you have done any OpenGL ES or a large amount of OpenGL development, you will find LESS IS MORE. The worst thing about OpenGL (even though I love it to death) is that it is _horribly_ messed up by extensions. Especially when people dont even realise you DONT need them to complete a task. The point is simple, keep your design simple, and clean and it will work. Add in more and more extensions, and API's and so on, and you will find yourself having to rely on other peoples code to 'do things right'. THIS IS INHERENTLY BAD. People might think these tools 'save them time' but in the long run they introduce dependancies - dont get caught, it will be better in the long run if you have more control, especially if you really dont need the API (hence why glew makes no sense). Cg is the only OpenGL addon I would say that is fair to break my rule. Primarily because it is the best way to get the best out of a platform's HW, which is an entirely separate thread. Having worked on projects with millions of lines of code, and some big nasty sims (on SGI OpenGL machines) I am only pointing out the problems I have experienced with such API's. Dont take my word for it, find out yourself if you think otherwise. :) --------------------------------------------------------------------------------- 2. Simple 3DS Mesh Loader Ok. Back to code. Heres a link to the dll and code: http://users.on.net/~dlannan/Files/3dsModelLoader.zip You only need the dll, but the source is all there to play with. Make sure you put the dll in a path where BlitzMax will find it - I use the project path where the exe is built. Its up to you however. Now, the Blitz Stuff. This is basically the same as extern for a Win32 dll. Note: Added a couple extra win32 funcs that are needed for the external dll. The last three lines are an example of usage. Pretty simple eh. Now you can go nuts loading in all sorts of 3DS models - if you are a Google Sketchup fan (or maybe you will be now..) it exports in 3ds.. and there is a huge amount of 3d resources on the Google3D site, well worth a look. Heres a link to a Skyline from Google3D exported as 3ds - remember to check any license issues before using it yourself. http://users.on.net/~dlannan/Files/skyline.zip Now. One other thing. At the moment, the normals are not being used. You can edit the dll code to enable this, but I will add a function for this later on - it will be important for lighting and smoothing. Next time I post will be fun time... I will try and pack designs 3 and 4 into a single post. Its going to be fairly basic, but its important to get why you want to do this :) |
| ||
| Awesome, a 3ds loader. I was going to code my own little model loader, but since you provide the dll and the source, I'm going to play with that a little bit. Thanks a lot (Also for the hint with glew)! |
| ||
| Ok. Rather than just design points 3 and 4. Heres the majority of it all :) I'll go through each section bit by bit then provide links to download all the goodies (bmx's and source for updated 3dsModelLoader.dll). --------------------------------------------------------------------------------- 3. General Object/Entity Manager (with simple scene capabilities) Ok. Why do we need this? Well, one day or another you are going to want to organise your scene or your 'world'. The organisation method you choose is entirely up to you but a good way to tackle this design problem is to make a generic 'object handler' or manager. This is an example of such a thing. The manager's main goal is to provide a _simple_ way to retrieve and store models, and other data in a single object structure. The object should be extensible - you should be able to later add scene graph management, ai management, collision management, and so on. So think carefully HOW you do it. The method I choose these days is referential data indexes maintained in the object. Ok, so whats that? Simply put the object or core generic structure that holds information never holds ANY information directly (there are a couple of exceptions, but will explain later), the information is always inferred as an index, a handle or a crc. The main reason for this is the ability to _easily_ extend an object without causing ramifications in other systems. The only data I store in an object is position and rotation (since these are core data used in almost every system possible - your object is essentially SOMETHING at SOMEPLACE). Referential data however may include things like: - Physics Object id - Model Identity id - Scripting id - State id - Type id - Animation id Structurally this is all fairly important, because the data size for your main object is kept fairly simple and small, while it allows you to 'attach' very complex functionality or information. By using handles or references you can then make the 'system' that issues the handles only worry about what it needs to do. Traditionally many games companies have gone C++ OO type routes for object creation, making an extremely convoluted and nested hierarchies of objects - this is a major problem when trying to extend any object. Even with advanced pattern designs you cannot easily make a similar construct with a heavy OO inherited system. Keep it simple.. and you will be happy :) So.. the GenericObject.. lets have a look at it: Fairly simple. Fairly basic. There is a trickier way to handle the properties (handles) of the object and that is make a properties map or vector that can be expanded, but I will warn against doing that. You will run into serious performance problems when you use complex data lookup routines inside your core entity - especially when its not _that_ necessary. Extending the object as it is will not effect any of the systems, and thats more important. Now for the object manager. We need a couple of extra types a PassType and a CombineType. You can probably guess what these are for :) PassType is to define a pass of how to render your objects in your manager. CombineType is how to combine texture outputs to generate another texture output or a screen output. Heres the manager itself. Ok. There is alot going on in here so I will describe the basics. The manager has an objectlist of type GenericObjects. When we create the manager we prefix our maximum object use (this is for simplicity sake not because you cant have dynamic lists :) ). You can then add objects to the manager (which are currently assumed to be models - this would change to be able to use physics models and scripts etc). Then you can add passes and combines to the manager which will be automatically executed when you do a RenderAll. Anyway. Thats the main part of the object manager. The main Render Loop resides in the RenderAll call - this is not always done this way but its more of a convenient function to keep it fairly easy to understand and use. Heres the bmx's of all the necessary files - theres no movement as yet just render passes. I'll discuss the passes, combines and main.bmx in more detail next. As well as show some more shader fun... like blur.. and so on. http://users.on.net/~dlannan/Files/gldemo.zip And heres latest update to the 3DSModelLoader.dll and source http://users.on.net/~dlannan/Files/3dsModelLoader_01.zip From this little setup, you will have ability to do a full deferred renderer. THIS IS NOT the best way to do it (normally use VBO's etc) but its a simple and clear example of how it works. I'll advance it to use VBO's down the track, so it ends up fairly decent. It will also get nice Cg shader's and such as well (Sorry ATi fans.. future installments using Cg wont be so happy on those cards). |
| ||
| ati hd3470 card,bmax1.30 error:Unhandled Memory Exception Error |
| ||
| Run in debug - should show where the error is. Make sure you have path with the loader, and the shaders. As well as downloading the skyline.3ds file (thats the data file Im using at the moment). The link to the skyline is in the earlier posts btw. A little side note, you can put any 3DS model in (without anim) and it should load mostly ok. There are some 3DS files that wont - so just be warned not everything is guaranteed to work with all 3DS files. Also, to play with rotation, initial camera and so forth you can use normal OpenGL calls to do so. However in the next installment we will extend the 3DSLoader to cope with Cameras. This is a _very_ useful thing for designers out there, and I will make up a simple 'room' with camera objects that we can use to move around our view with. The Deferred rendering at the moment is purely albedo + a very base specular. By default you wont see much, just the back of the skyline :) Heres what you should see:  <Edit> Just updated gldemo - wasnt clearing depth between passes. This is now modifiable (some passes may want depth left untouched). So make sure you re-download zip. |
| ||
| Now for some more discussion on topics 4. Render Loop 5. Render to Texture Passes (albedo and specular) 6. Combine Passes (with output pass) --------------------------------------------------------------------------------- 4. Render Loop The main render loop should be as simple as possible. But, added to that it should be flexible enough to achieve any render output needed by a designer - these requirements sometimes seem at the opposing ends of the coding spectrum :) With deferred rendering though, there are some nice things that allow us to achieve a substantial amount of both these requirements - simple, and highly featured. The main trick is to look at using lists to manage how you use your objects. For instance, dont try and render all the objects all the time. It makes sense to spend a little bit of CPU power putting objects you _need_ rendered into a list, and the others not. Then render the smaller list. These are the sorts of things I will leave to the reader to add an improve upon. As it stands its not a core need for the example of a deferred renderer. --------------------------------------------------------------------------------- 5. Render to Texture Passes (albedo and specular) The two lines of: gObjectManager.AddPass(0, 0, 0) ' albedo pass gObjectManager.AddPass(1, 1, 1) ' specular pass Are quite handy ways to add passes into our system. These passes are being applied globally and each pass refers to its own shader and texture output, giving the programmer a wide range of pass capabilities. The complexity of the pass comes in the use of the shaders and the combining of texture outputs... to be discussed in the next post :) |
| ||
Seems if i compile and run the code in the ide I get the same error as dream, yet if i then run the compiled exe itself i get a screen like your shot above. debug log output from inside ide is Executing:gldemo_main.exe All Ok All Ok All Ok All Ok All Ok All Ok Making Texture: 255 102 51 Making Texture: 255 255 255 Making Texture: 170 170 170 Making Texture: 226 226 226 Making Texture: 30 30 30 Making Texture: 9 21 181 Making Texture: 40 40 40 Making Texture: 183 183 183 Making Texture: 0 0 0 Making Texture: 181 181 181 Making Texture: 114 114 114 Making Texture: 255 255 0 Making Texture: 73 73 73 Making Texture: 60 104 142 Making Texture: 153 153 153 Making Texture: 0 0 0 Unhandled Memory Exception Error Process complete |
| ||
| From the log you are running in release, so you will find it hard to determine the line where the problem resides. In debug the line should highlight when you get the exception (Program->BuildOptions->Debug Build). Which should tell me which line is the problem. I'd suspect its more likely to do with a missing file somewhere - I'll rerun on another PC to check. I have added my build of the 3DSModelLoader.dll to the gldemo.zip now too, so you can try using that too. |
| ||
| Finally a bit of discussion about the combine pass, and what its doing. --------------------------------------------------------------------------------- 6. Combine Passes (with output pass) With Deferred shading or rendering systems the key to good results is making sure you composite textures together in the right way to produce a desired result. Some combine shaders are quite complex, and apply features while combining textures for an output. This implementation is relatively simple all it does is multiply the incoming Tex1 with Tex2 and output the result to a fullscreen quad. Of course it all depends on the type of output you need for a display. Heres an example of using a different operation than MUL - this uses ADD in the fragment shader.  With shaders the power of rendering is at your finger tips. You can produce amazing effects with quite trivial changes to the shaders you use. In the next few posts I will explore the world of ARB shaders and just what you can get to happen with them. |
| ||
| It fails on the line gObjectManager.RenderAll() in your main loop, i have redownloaded the files and its still the same. |
| ||
| Hrm. This is odd. I have run it on an ATi lappy (very very poor GPU) and seems fine on that too. I'll try a couple of other PC's see whether I can reproduce it. Im pretty sure its not the shaders, since I generated the ARB from ATi's own ASHLI viewer, so if there were any problems I would would expect them on Nv cards :) It may be related to BlitzMax too, Im running 1.28 - I'll update and see if that causes it. Have you done any modifications yourself? Are you using the same data set? Have you modified the shaders at all? etc.. need to know exactly whats different - if it were hardware, youd see problems in the compile stages. Also make sure you use the bmx's and shaders from the last gldemo.zip link. <Update> Ok. 1.3 BMax works fine too. Will check on other machines. <Update> Have changed a loop in the RenderAll, but that shouldnt be the cause of any problems, feel free to try it again though. Also if you have gldemo_objectManager.bmx open you should be able to see what line in the RenderAll function this is occurring. <Update> Have tried on 4 machines now (ATI, Nv and Intel GPU's), and cannot get this to occur. If you can get me more information about where in RenderAll it is crashing, please let me know. Apart from that Im stuck on this problem. |
| ||
| Ok. Continuing on, I have added a couple more shaders. An edge detect type shader (just very basic) and a second composite fragment shader (MUL and ADD composites are now separate). The new download contains everything needed: http://users.on.net/~dlannan/Files/gldemo_01.zip Heres the result:  Next post I will add the camera controller, some movement, and a pass that gives us things like bloom and blur. I'll also change the scene to be an indoor view, so as to show of the effects more fully. |
| ||
| The new version is giving the same crash when compile & run from ide are used, the line in the object manager is RenderModel3DS obj.model_id inside the passes loop, the obj.model_id appears to be pointing at a value so I am not sure why its crashing, but again if i then run the compiled .exe it works fine. bmax is 1.32 and i am on vista 32bit with nvidia 8500gt, its a wierd error tbh. [Edit] Well I just tried compiling the code in a version of the maxide I compiled from the source using fltk and that works fine, so the crash is from the version of the ide that comes with the latest update. |
| ||
| Works fine here and looks awesome. You're updating this tutorial real fast, it's hard to keep up :) |
| ||
| joncom2000 - thanks for that. Sounds like an obscure little bug. I'll see if I can lock down memory accesses to see if it improves. Are you running on a 64bit CPU? that may well be a contributing factor? If you are, I'll add padding to the structures to ensure alignment (might do this anyway just in case). Next update coming tomorrow morning :) .. Nice indoor scene, with camera and shows off the passes and combines a little more. I'll also get into moving it to VBO's - perf is generally much better with them. Oh, and will start talking about uber shaders :) .. ie shaders that might do more than 1 or more passes in a single shader. Cheers.. |
| ||
| Ok here we go. Things get fairly interesting from here on in :) Over the next few posts we will visit blur, bloom, hdr, shadows, and lights (not necessarily in that order :) ). In traditional rendering these effects are normally non-trivial features to implement. It usually requires specialised pipes, carefully sorting the objects in the scene, and making sure other shaders dont clash with the new effect shaders. With Deferred rendering, this all becomes far easier, and much more able to be easily implemented without conflicts. Here we are with a new data set, and roughly the same setup for the previous examples. Heres the code and data: http://users.on.net/~dlannan/Files/gldemo_02.zip Heres roughly what you should see once running:  And heres the new 3dsmodelloader (now with steak knives..) It handles a wide range of texture types now, and additionally has fixes for some bugs that were from the original code. http://users.on.net/~dlannan/Files/3dsModelLoader_02.zip After a quick glance at the code, you should notice not much has really changed - some minor changes in the shaders, and some small additions in the passes, thats it. I havent added camera yet (we wont need it for a little while yet.. but I will add over the next day or so). So.. Blur.. how about some blur eh?!! Well theres all sorts of blur we could do - camera focus blur, motion blur, gaussian smoothing blur.. etc. I'll show you how to do a simple motion blur (camera focus needs a camera :) ). What the blur shader does: Takes the same shot, shifts it slightly, softens the merging output and repeats a number of times :) I can see you thinking.. see if you can get blur in before my post of it running :) |
| ||
| Doh. Sorry guys. there will be some delays (probably a few days again). Have some more sim work to sort out. Will be back as soon as I can with the rest. Cheers. |
| ||
| Look forward to seeing more, good new is the last version doesnt give me the error anymore so I can tinker with the code and keep retrying the changes :) |
| ||
| Don't worry, Grover, you've already done a wonderful job - we'll wait :) |
| ||
| This is very cool. You might consider using 'superstrict' rather than 'strict' to take better advantage of optimizations. |
| ||
| Thanks GW - will do (not having done a tun of Blitz for a very long time :)). |
| ||
| Hey Grover, First: Thanx for this tut. Just the hints i need to complete my own project. _Nice_ work, great idea. Second: U seem to be a real (relative) ass-kicking pro in (graphics) programming, so.. just interested in it.. why are u using bmax? Its just that i personally need some args towards my own team why the hell we use bmax; not that i havn't any, but hearing urs might be really interesting.. Coke :) |
| ||
| Hi again - popped in again (still busy with a sim), thought I'd answer yer questions BeAnAge.. Have been burning code since days of Vic20 and C64 :) .. have been involved in all sorts of industries, from industrial plc programming, games programming (worked with Ratbag, Pandemic Studios and a few others) and have settled in simulator development as a principal engineer at Sydac.. doing rail, car, truck, excavator and all sorts sims. Why bmax.. hehe thats a funny q actually. It was a mate who I had a chat with whom is into blitz (and a few other similar hll's). We talked about how to produce good gfx results in blitz, I assured him that with a simple deferred renderer you can do it in almost any language. So there is why bmax for the tute, and a wave to Toby and Paul :) Whether bmax is good for full production is a tough call. There are some things happening in hardware in the next 12-18months that make it a relatively good idea. But it also makes other hll (high level languages) worth looking at too - java, python, perl, C#, Lua etc... Intel will have an 80core cpu next year, and NVidia have their 480 core units out now. Whats this mean? You will have more processing power locally on a single machine than you will be capable of fully utilising :) Thus, the problem for developers becomes not "Oh.. must spend 10x effort in performance coding" to .. quick.. best hll and get the gear out the door. This is the future of game development people - beleive it or not, this is where Intel, AMD,IBM and NVidia are all pointing their dev too. So, the good things about bmax, is it provides a very rapid development environment. It provides it across multiple OS's/platforms (although I think thats going to disappear too), and it provide decent debugging and extension facilities. Everything you need in a HLL. For development purposes, look at what can build quality output, with ease of creation. Tools are key nowadays.. Put simply, if you can get a game out the door X times faster in blitz, and still have a nice deferred renderer, why would you bother with a low level lang? Its the same reason why asm has pretty much disappeared from PC dev. My personally fave language at the moment is Lua - beautifully simple (20 ops.. simple interfaces.. extremely powerful). But again.. HLL wise.. theres a huge amount to choose from. One thing I always harp on about to my team though, is its NOT the language that makes the application/game/sim work its the design and architecture that is critical to a good result. If a particular language provides fast dev, easily extended, and so on and it fits your needs dont think too much about it. If you get into language wars, then seriously the people in your team need a rethink about WHY they are doing it. :) Language should never be a specific reason for a development path - we use 4 languages in our sims for example. Java, Lua, C++ and some Excel stuff :) We use the language where it makes sense, and saves us time, money and dev. Hope that helps some :) And good luck with it all. Cheers, |
| ||
| Grover, That is some good insight, thanks for sharing. |
| ||
| Thanks for the wishes and fast answering, indeed interesting, especially what u say about hlls & architecture; fully agree with ya. Using max to code a free blackbox game creation framework as a so called "special learning performance" for my hs degree (ger. "abitur") together with some other guys, we decided using max for two reasons: _First, it has a great learning curve, and makes it really easy for beginners to make it from spaghetti-goto-programming to advanced oop. _Second, its a real rad (Rapid App Dev) tool, right what we needed to get confident in being able to make the whole thing in two years^^. And yeah; finally its organic and dynamic archiceture deciding about an apps success, not the language. hmmm.. but some languages make some designs easier than others, do they? (We began some prototype programming in bplus, it was a desaster :) ). _EIGHTYCORE_ cp unit, WOW, thank god i'm coding multithreaded :D .. This and the possibilities in AI and Physics programming opening that way are just unbelievable .. you could have a an ulimately trained NN, highly resoluted voxel sims, cloth, etc. etc. .. <------------------------------------------------------------------------> But comming back to the tut :).. why arent u using glslang? Just a little too lazy to learn all those arb_fragment_program assembler style instructions, but now i propably wont get around it :| .. anyway, great stuff. Coffee. [Update:] Its about your comments on double usage. When using bounding volume hierarchies a.k.a. a strong scenegraph implementation, so coords are always relative, u should keep being able to use floats or even ints for independently large scenes, shouldnt ya? |
| ||
| Back again. Should have an update tomorrow. BenAnAge: Some languages are restrictive, and some are less so. But its generally the constructs used in languages that limit architecture and design - the core of developing fast, and nice quality applications. An example of this would be OO patterns, generally seem like a good idea but there is a huge amount of misinformation about OO being good for extensibility - ask any long term OO developer, they will all say the same thing OO type languages lock you into hierarchical structures that are generally almost impossible to extend at later dates. Hence why my warning about focus on design, not lang. If I could, I would probably write all our sim's in Forth or a FBP lang, but the problem is training so you have to work with C++/Java etc trained people thus limits your choice also. One thing you can do is use classic generic flat interfaces for your systems, minimise your embedded 'knowledge' in an object, turn it all into data (function and data is data.. if you plan carefully :) ). An example is how we generate code from Excel templates, thus code (function) is now a simple data entry in a sheet - this is the way you minimise dev time, good tools. Ooh. Multi-threaded, and multiprocessor.. beware the comparison!!!. Threads ARE NOT processes (unless you are using Intels TLib which is a lib to make threads like processes). A common misconception is that threaded apps work great on multi-cores - this is a plain wrong assumption. In fact thread switching to another core usually costs more in context and resource changes than keeping it on the same core. Most cores come with 2 or 4 hardware threaded pipes and thats the best way to utilise threads - locally on the same core. For multi-processing, its a little more difficult/design required - especially if you work with something like the PS3 Cell (8 cores..) there needs to be a 'smart' interprocess communication method. We use IPC for our sims.. but are looking for better ways to do this anyway (theres all sorts of solutions, its early days for this gear). But yes.. the future of development for all PC dev is going to be quite amazing. The type of things you will be able to do on a single PC will skyrocket. So implementation interfaces become critical in developing apps with minimal bugs, and quality results. Take a look at things like Matlab and Labview theres alot of promise where these tools are heading. Removing the need for real debugging, providing robust 'black box' components and leaving the design up to the application developer... I see this sort of thing become quite common (even in game dev). Imagine visually gluing your data together to make your game - spend less time of tech generation and more time on the game design component. Thus.. better games.. too imho. Who knows.. will see :) Ahh glslang - the main reason for this is compatibility across GPU platforms. ARB is a fairly fixed standard on a _large_ range of cards ( ATi, Intel and NV). Also, there are a number of tools you can use to generate it (ASHLI Viewer, and I think RenderMonkey etc can export ARB compatible shaders). And most importantly is simple interface is something everyone can understand. The focus for this tute isn't on the shaders themselves but the implementation. I will move onto shader focus later but that will then jump to Cg: by far the best of the shader compilers.. and the best for tools and dev, can even generate ARB from Cg too (even for ATi cards). For me glslang was too little too late. Remember on PC the majority (good 30% plus) of your audience uses Intel GMA chips.. this is well worth remembering when choosing technology btw. http://unity3d.com/webplayer/hwstats/pages/web-2009Q1-gfxcard.html Sometimes easy to ignore the mass market.. but thats where the cash is if you are business minded :) |
| ||
| It's sad that the 3DS loader is DLL only and Windows, right? |
| ||
| The 3ds loader is correctly a dll so restricted to windows but the rest of the code should work on linux and mac with a few minor changes, I posted a version of the initial code that used glew so as to remove the getprocaddress calls for the gl functions, this should make it possible to use the rest but you would need to provide your own model loader or you could create a simple scene directly in code for test purpose's. |
| ||
| @Grover Ok, concerning hlls its propably just a question of what you are heading for with your app. When its a simple monolithe game, a more restricted language might even figure better than a more open one, eventually eliminating the need for coding systems whose implementations are implicated by your apps genre. But for programming a complex framework archtitecture, the bmx OOP came/comes really handy in some/many/most cases :). Sry, realising i lied. I am not coding multithreaded, or at least not in the proper meaning of the word :). Actually at the moment i'm just designing multiprocessed. For our framework, we plan to implement / are implementing a solution, where the renderer, physics, and ai run in different processes, all chewing data from the scene graph shared into sm via file mapping, communicating on events, pointers, apptime etc. using a tcp pipe. Hence you really got some exp in that field.. what do u think about this..? Is there anywhere kinda fragment/vertex program <=> glsl wrapper out there? If not, eventually one could write one.. see if i got time for it :).. would it be worth to put effort into something like that? |
| ||
| nawi: The 3DS Loader is actually crossplatform compatible. However, the dll usage for blitz is not. This is a demo, so Im not going to spend tuns of time on different platforms, sorry. But if you compile it as a module, it should work with some minor changes. BeAnAge: Ok. Will explain again. bmx is fine, but OOP is NOT. Having worked on professional projects for over 20 years, I have never once seen an EXTENSIBLE OOP implementation. This is a design limitation of OOP itself! Not the language, but the constructs of OOP. As soon as you encapsulate an object you are doing one _very_ bad thing, you are stopping it being able to data _communicate_ to other objects without adding complex interfaces, or embedding into intermediate objects. Either way.. hierarchical object structure is programming wise _wrong_. It seems like a good idea, and at university they tell us it is, but practically its bad. You will find any project lasting a year or more with an OOP base design will be totally inextensible 5 yrs down the track - which in programming terms is shit. We should be able to reuse ALL the work we do. At the moment in the development world, we reuse as little as 1% of the code we write. You'll find this true as you go on in the development world - I myself, have written same routines for different companies many times over. So beware the OOP.. think very carefully about how you use it, and its depth of structure (usually the main reason for inextensible code). Multi-proc through tcp is entirely doable - beware that you will see coalesced data on the comms and need to cater for the latency that causes. You _should_ _not_ turn this off (Nagle bit). You need to ensure you send enough data in a cycle that it keeps pushing the data through. Shader wise.. use Cg. Yes its command line, but have a good look at it. It can do all sorts of conversions, optimizations and so on. Additionally, if you get keen you can use CUDA to make shader code run on your GPU - this is really where NV have it over everyone. Running 'general' code on a 260core.. makes it 'fly'. We ran our own physics system, plus anim, plus scripting system (yes whole vm on a gpu) and the performance is insane. Would recommend moving that way, since most ppl are :) Update coming... |
| ||
| @Grover Aah think i got ya point.. when designing a complex OOP project, you create large inheritance structures.. mostly based on one (or a little amount of) base types.. this can save you motherloads of work, but find one simple semantic error in the structure, or just get having some parts of this derivate would figure far better to an upper base type, and you run into serious troubles.. you are right (if u mean that).. and it makes me a little nervous, hence our whole project is based on a really large system of inheritance hierarchies.. gotta think about that. Thx for the hint about it. Seems to be one real problem with Nagles Alg. Cuda.. mmh.. in the past i kept avoiding it, mainly for its commercial background (same for cg; ok, anyhow everything is commercial, but cuda [and cg] is nv only.. just dont like this monopoling). Thought about an OpenCL implementation of some functions, but ocl is far future, isnt it? [Edit:] Ok, right about to thwrow my principes away.. damn, just watching myself downloading the toolkit :) |
| ||
| BeAnAge: Yeah OOP is fine for 'one off' type technologies. Which is quite common. But since the scale and size of code is becoming substantially large and we can no longer afford not to resuse code - this is where the OOP paradigm fails badly. Even in a simple single hierarchy it can become the worst of nightmares to extend a class to 'do more'. If you are keen have a read about FBP - its an interesting way to tackle the problem: http://jpaulmorrison.com/fbp/ There are alternative ways too - get and have a good read of design patterns for OOP. There are some effective ways to 'help' the problem (although imho there is no solve in an OOP lang). Yeah. CUDA is gathering a fair amount of steam. Even ATI are moving towards a similar lang (although their drivers on linux suck bad, and their lang is generally poor). For a product to the door - dont concentrate too much on CUDA, think about multi-core use as your main target. You gotta fight the fires you can put out. Think of CUDA as a Voodoo like extension, eventually everyone will have something like it so if you have 'some bits' that can use it, its a good bullet point to put on yer box :) Especially if you can do voxel physics realtime :) Beware expending too much effort on unique tech dev - if you can get some good tools to help, do it. |
| ||
| @Grover FBP is a real fascinating field of dev!! Just read the article .. Think about a design, where you have.. maybe a about hundred individually working processes, "talking" to each other, forming kinda .. "process society" (sounds odd, much like a "Matrix" thing^^).. Unimaginable how powerful such an architecture could be! If you have a new version of an alogithm, just throw it into the network as a new process, and it will tell the old one that it's no more responsible for precessing data of this type/ with this signature / of that size .. but they would all share the same communication protocol and data analyzation systems, wouldnt they? The next thing then would propably be, to link the data type analyzation with a NN; mind-blowing.. Mmmh.. at the very moment, we do all our (really experimental) voxel calculations via a provisoric use of shaders and fbos and lod.. its all still really in development, and many is just ideas we right implemented from scratch just with the functions we found right in front of us :).. CUDA or any other gpu env might give it a real kick.. |
| ||
| BeAnAge: Its a damn interesting time for computing thats for sure :) .. The next 'big step' will be the thing that solves these massive parallel programming tasks with some easy visual programming toolset. It will steamroll everything once it happens... just a matter of time :) Imho alot of it will tie in with some sort of FBP process - black box development has to be the answer, its the implementation that will be interesting :) Beware though.. have been burning coding hours on FBP for 5 yrs now.. its fascinating research but lots of long yards to go.. |
| ||
| Sorry all. another trip to Melbourne for more sim work slowed me down again. So.. more updates coming soon.. :) |
| ||
| Ok. Back again. I promised a example Blur compositor/shader stage, and sort of forgot I needed something moving in the scene to make it worthwhile :) Soo.. after a bit of fudging some verlet blitz physics code I found here (Nate the Great - its yours that you posted here: http://www.blitzbasic.com/Community/posts.php?topic=80316#902983 ) .. with some mods!! I added a simple fixed constraint, so I could do a swinging lamp, which will also end up being a nice way to do shadows, lighting and glow/bloom effects later :) .. I also added some simple fps type controls (very simple) and some basic look around controls for mouse (again, v basic). The main part to look at is the 4th square on the bottom right - this is where motion vectors for scene objects are collated and then used to determine how much blur to apply to an image using NV's paper on it : http://www.play3d.net/books/OpenGL%20Shader%20Tricks%20-%20NVidia.pdf Its really not that amazing.. but shows how you can quite easily add new effects/passes/composit methods to a deferred renderer, whereas in a traditional renderer it may be quite a bit less easy to do. Screenshots and code coming tomorrow (tidying it up a bit - yes.. bit messy at the moment, rushing to add 10 things to show one effect :) ). |
| ||
| awesome! Looking forward to it. |
| ||
| Yep, I'm also looking forward to this. 'Has been real interesting to follow this thread. |
| ||
| Ok heres the next installment. The Blur Shader isnt in, because I wanted to first cover what has changed to allow me to use a Blur shader :) This is it. And heres a changelog so far: 1. Added Verlet Physics manager (very basic but easy to improve) 2. Added Custom objects to use (allowing overrides for Render and Update based on types) 3. Reworked the Object manager to handle custom types 4. Added facilities for custom rendering within the deferred renderer 5. New model (lamp 3ds) 6. Utility CRC function (most handy... :) ). 7. Mouse + WSAD controls for movement http://users.on.net/~dlannan/Files/gldemo_03.zip Heres roughly what you should see once running:  Tomorrow I'll put the Blur shader in as well, with some enable/disable toggles so you can see the difference. Meanwhile have a play with the physics see if you can make a simple rope from multiple constraints rather than just the one that I included. Later this week I'll attach the lighting to the lamp, some shadows and more post effects like bloom, and such. |
| ||
| Unfortunately I don't have the time to play around with you last tutorial, all I can say is: thanks for doing this. :) |
| ||
| This is great! Very nice tutorial, Grover! I'm having a few issues with the textures (apparently the lower the mip level, the worse it looks from what I could tell). Here's a screenie:  My specs are in the sig. |
| ||
| Wow.. thats a way kool effect :) I suspect thats something to do with the ARB shader / texture setup. I'll have a look, I suspect its a pretty simple fix. It may even be something odd like texture loading. Your machine appears to be 64bit - so I think its probably memory alignment. I'll try a couple of things to see if things work better. :) |
| ||
| Yeah, 64-bit. Forgot to mention thet :) Yeah, it is kinda cool :D The further I get away from objects, the more wacky they look ^^ |
| ||
| Nothing new? Been waiting for the next great tutorial. :) |
| ||
| Will SSAO come? |
| ||
| hey grover nice tutorial and thanks for using my physics engine... :) Meanwhile have a play with the physics see if you can make a simple rope from multiple constraints rather than just the one that I included. it should be very easy to make a rope but my physics engine is admittedly a complete mess. Im not sure how you managed to understand it enough to convert it to bmax, it was my first one. |
| ||
| Hi guys. So very sorry for the long delays. Too much to do, no time. However, expect updates in the next few weeks (I hope). I have vector motion blur and some ambient shadows (yes SSAO - its actually quite easy!!) etc on the drawing board :) Nate you are welcome to the source (its in the zip) should be fairly easy to port as needed. Adding a proper rope (with hinges and such) should be fairly trivial :) Have written/used many physics systems over the years.. so wasnt too bad to wade thru it :) So.. thought Id bump this to let you know I havent forgotten - sims have kept me very busy.  Control room for a current sim being developed in Melbourne, Aus. |
| ||
| Awesome, absolutely awesome. I once heard, unknown source, unknown time, unknown reason, about a 4M Pixel output limit of current graphics cards, which prevents the resolution of monitors to exceed that certain limit, interrupt me if i talk crap.. especially triple monitors with each monitor resoluting > ~1280*~1024 .. so what are these on your image resluted to? Just interested.. |
| ||
| @BeAnAge: I think you are restricted to 1 device for that resolution. You can only have 4096x4096 as a resolution. However, you can have more devices defined. (if someone knows better, please correct me) I've seen a few demos around the OGRE forums on more than 4 screens (both OGL and DirectX 9). Last, but not least, there's this |
| ||
| Do you think there could be an advance in graphics realism, if you render at such an extreme resolution, and scale the result to monitor res? That would be some Anti-Aliasing! Ok, it would be a waste of performance.. but I had the idea :D (no, actually not, its called Supersampling... LOL) |
| ||
| Any news? ;) |
| ||
| @BeAnAge: That's what Supersampling AA does :) Currently, most AA implementations are Multisampling (only rendering the borders of meshes at higher resolution), however, the nVidia control panel has a setting for supersampling AA, and the new ATi 5000 series also have supersampling AA support. Ontopic: What the above poster said :) |
| ||
| Hi Guys.. man.. the above sim has drained my life :) We have gone a bit nuts with features (SpeedTree, PhysX etc) and has cost a fair amount of dev time. For the questions: - The above sim, takes 1600x1200 x 4 channels. All channels are split to have an output to both a LCD like above, and a projector setup. - Gfx wise we use a rather hi end setup from NV. FX Quadros - FX 4800 or greater. Each channel has a separate PC to drive it, and our technology synchronises it all, and internally manages data movement around the system. - In the sim business there are a multitude of solutions for spanning channels out to multiple devices. You can use things like Matrox TripleHead2Go to drive the output (one wide set of channels out to multiple DVI outputs). Alternatively you can run like ours where each pc is synchronised (this is much harder but more flexible - and is only internal software solutions, we cant apply it to other apps/games/output). - resolution wise, there are some interesting problems when reaching for high resolution outputs in sims. One of the major problems is memory and internal GPU speeds. As the res goes up so does the texturing size, so does mem use, then raster perf needs to be higher etc. These days we use gaming cards like GTX295 and specialised NV boxes with (2-4GB on board). These are the current best ways to tackle the problem, and likely to continue with solutions like Tesla, and similar. We currently steer clear from ATi solutions due to the serious problems we have had in the past with their fog/alpha implementation and their extremely poor Linux drivers. They seem to be improving in the alpha/fog area (recent shader setups we have tried have been better), but in the Linux area they are miles behind NV. Added to this, we are moving to GPU based physics/motion simulation systems so CUDA/Cg/OpenCL are the main choices for us as well. Now.. I have two weeks off. And yes, that means updates!! :) So very sorry about the slow going.. but everything is taking my time at the moment.. Some immediate updates though. I found a minor problem in the 3DSLoader, so heres an update. If you were getting crashes when trying to use different paths than the demo.. this will fix that. http://users.on.net/~dlannan/Files/3dsModelLoader_03.zip |
| ||
| Finally! |
| ||
| Ok. First update in a while is fairly simplistic. I have added an SSAO method as described here: http://translate.google.com/translate?u=http%3A%2F%2Fsteps3d.narod.ru%2Ftutorials%2Fssao-tutorial.html&sl=ru&tl=en&hl=en&ie=UTF-8 Heres the source and an exe to play with: http://users.on.net/~dlannan/Files/gldemo_04.zip For people interested in how to get glsl or many other formats into ARB, have a look at the NVidia cgc compiler. It allows you to generate quite generic ARB code (this means quite compatible across hw too). heres a screenie:  For a basic round up on this update. This SSAO doesn't implement the Blur and it is fairly simple, but extremely fast. Like most SSAO implementations, a number of rays are processed to determine depth proximity of pixels. From this area is shaded based on this. If you use this in other scenes its important to understand that the shader for the SSAO will need to be modified to suit. Params c[0], c[1] and c[2] need to be modified if the scene depth changes (c[1] determines depth).. ray radius is used for the shade calcs (c[0]). The bias 'tweaks' the way the shading is implemented in the scene (depth related). I'll be adding the Blur pass, and improving the Depth collection (at the moment its a hack). Its fairly easy to add yourself if you feel keen. :) This blur pass will be the basis for a fullscreen 'bloom' pass that will be done in the next few days as well. Hopefully I will get to implement the above intended features into this little deferred rendering system. I should point out too, there are _tuns_ of ways to do many of these things so be sure to have a look at different solutions or even come up with your own! :) By the end of the week I hope to have some tools to help out too - allow you to design your own passes/combines from a simple app, then load them in as an asset. This way you can make your own multi-pass deferred renderer to work the way you want it to. Oh.. from the screenie, there is a funny coloured texture in bottom right texture slot (tex4).. this is a nice rotation matrix kept in a bmap. Again this was taken from the above post/method. Well worth reading about if you want to know more about SSAO methods - I also have a tun of other links/info if people are interested. |
| ||
Oky dokes. Heres the SSAO with a bit of blur (Im going to replace the blur shader.. its just too meagre). But starting to look nice :) And the code - all bmax still. http://users.on.net/~dlannan/Files/gldemo_05.zip Some minor changes/additions - ability to set the 'save' texture for the depth buffer during any pass (this will be useful later). Also, added texture filter setting - currently nearest or linear but can add more as needed. This is important for data textures where you dont want filters applied (select nearest). Next to come is some vector motion blur, lighting (attached to swinging light) applied in a pass, and bloom which should pretty much wrap this up - otherwise will be here forever adding features. Oh.. if there's some sort of feature people are specifically interested in, then please comment. Most things in a deferred renderer tend to be fairly simple to do - even AA isn't as bad as its made out to be (simple edge blurring pass is usually enough :) ). <edit> Oh sorry.. I will add a shadow pass as well with the lighting. Something simple.. but hugely effective. <edit> And beware that the SSAO shader needs a decent video card to perform correctly. If you get a shader error about too many instructions, then the hardware you have isn't capable. and if you are a keen OpenGL fan.. I'll put up my webby (which is currently under rebuild) for OpenGL + Lua driven stuff that people might like to play with. NOTE: Sorry, but the shader has a bit of a bug in it. Im using different texture sizes, and need to compensate in the shader algo for it. So there are some 'odd' artefacts in this at the moment. Expect more tomorrow :) |
| ||
| Some quick comparisons from using models on the net. I have updated gldemo_05.zip to contain some extra key presses. 1. Enables/Disables the debug textures on the side 2. Enables/Disables rendering with SSAO 3. Takes a screenshot into a screenshot folder. Heres the Temple of Castor and Pollox without SSAO _______________________And here it is with SSAO turned on.   Makes quite a stark difference. I have tried a few other models from Google 3D Warehouse and its quite surprising the results with this 'simplistic' SSAO technique. I suspect adding some lighting and shadows should result in a very nice looking scene. :) I should point out - currently the way things are being done is very 'heavy' processing wise. So you may see quite substantial frame drops if you start trying scenes of 'millions' of polys :) The above scene has 300,000 polys and remains a solid 60fps on my GTX260. But I doubt this will be the case for all cards :) |
| ||
| Another test I did (on non building object) was more interesting as well. See how even the dark components of the scene are improved. Now its time to add a light. Without SSAO _____________________________________________________With SSAO   Also, I will add a couple of other useful things to the renderer. I have added alpha capabilities, but I want to show how to add an 'alpha pass' to the deferred renderer - this will be all important with more complex scenes. A couple of bugs were found in the 3DSLoader again as well. That is now updated: http://users.on.net/~dlannan/Files/3dsModelLoader_04.zip More tomorrow... bye! |
| ||
| Okay back now with a new feature - Lighting. There are a number of types of lighting in a scene, in fact SSAO is a 'trick' lighting / shading effect to simulate some of the capabilities of a Global Illumination (GI) system. We will come back to global illumination when we do some Caelum / Scene Ambient lighting. Other types of lighting are: 1. Point Lights 2. Spot Lights 3. Directional Lights 4. 'Special point lights' And there are probably many more, but these are the main ones you will deal with in 90% of all scene rendering. 1. Point Lights ------------------ These lights are fairly simple. They have a position, a range, some sort of falloff rate and generally a colour setting. These lights can be made in a single pass shader within the deferred rendering system - extremely simple and will be the first example of lighting. The main thing to remember about point lights is there is no associated 'direction' with the light itself. It makes doing projected shadows from them, quite difficult (especially if you have numerous ones). 2. Spot Lights ----------------- Spot lights are like point lights, they have colour, range, falloff and position. But they also have direction and usually some sort angle measurement for the angle of the spot light cone size. Spot lights are much easier to implement projected shadows and stencil shadows, and are more commonly used where lighting is important to a scene. 3. Directional Lights ------------------------- These lights are replacements for 'sun lights'. These are very distant lights that mean their lighting is a parallel wide source of light. These lights also work well with projected shadows. They can be tricky to 'get right' when combining with global illumination and other light sources. They often have position, direction, range, and colour. 4. Special Point Lights --------------------------- The only reason I mention these light types, is because they are quite commonly used, but rarely mentioned. Often a scene needs something to 'look like a light' to the camera/viewer but not necessarily need to be a proper light source. An example of this is a brake light on a car. Generally it is enough to have a 'billboard texture' or something similar take the place of a light and render a 'brighter' area for the specific feature. Another example would be the lights on a ship or an airplane at night. Additionally these lights often have their own falloff and scaling systems, so that even when they are distant they can still be seen properly, a normal light will become pixelated and end up too small to be rendered!! So these light types are commonly used in nighttime scenes. Ok, so thats the lights we will cover. Quite an amount to show, but I will try keep it simple so that things dont get too confusing. Code, Screenshots and more coming soon. |
| ||
| Aaah SSAO. YES! Although I do moan about the usage of ARB programs instead of GLSL Shaders, this Renderer seems to be cutting the edges of available free blitzMax 3D rendering so far indeed.. Thank you! |
| ||
| Trust me BenAnAge.. you will be thankful that you are using ARB. They will work on a much wider group of cards (I even managed to get the shaders running on some crappy mobile gma chips). As I mentioned many times before, its really not hard if you prefer to use glsl, to simply use glsl and do a compile pass with cgc to make ARB sets. I probably should explain a little more: All HLL shader languages get compiled to an ARB/ASM set of instructions _anyway_. So to work with that output is a good thing, since most cards see the code as the same instructions (or very similar). Sadly the multitude of shader languages hasnt helped improve _anything_. Theres no common toolsets, theres no common editing suites for artists, there is a general mish-mash of HLSL, GLSL, Cg, CgFX, and all the other 'special ones'. Its why I steer clear of any one language in the shader world, none are better than the others and none are worse.. they are just all damn different which makes it an utter pain for a developer. I should point out.. I do _tend_ towards Cg for most shader work on our sims and I also do this for games. Mainly because the tools for it are far better than any of the others (especially on Linux). But I do tend to compile all my Cg shaders down to ARB format anyway.. Point is.. everyone has their own way. And there are just tuns of different ways to tackle the 'shader problem'. Find a production pipeline that suits your team and toolsets, and stick with it. For this tutorial as I stated way back, ARB is mainly for cross compatibility and for commonality (and its reasonably easy to produce from other HLL's). I might put together another thread about this, so people can see its not really too big a problem to generate it. Hopefully I'll have a helpful tool for this gear in the next few days anyway. Allowing ppl to use GLSL, HLSL etc.. and generate ARB output. No update today.. having a break :) Points lights tomorrow. |
| ||
| Wow this is awesome. I allways thought, this topic was way to difficult for me... but thanks to you, Gover, Ive learned much about shaders. Can't wait the points lights :P |
| ||
| Very interesting read on the topic in general and the steps you've taken including their results |
| ||
| You know, Grover, it took me a few days of getting into openGL before I came to appreciate what you're doing here as much as I do right now! Unfortunately I ran into this as one of the first examples I found and was perfectly baffled outside of any hope to chow through this. Now it feels like the cleanest approach I've seen so far and I not only begin to understand it all, but rather enjoy your style of coding and find it closest to my own heart! ROCK ON! |
| ||
| Ok. Sorry about the missed update again. Sims rule my life again. Having to put together proposals etc.. keeping me on my toes. I however have a half built Cg shader tool that I think ppl may enjoy building ARB (or any type for that matter ) shaders in. Prolly a week or so away but dont worry its coming. The Shader Tool is really to help people with shader generation for use in this tutorial - yes, its grown a little bigger than initially planned, but hey I figure its better done with some 'fun stuff' than lots of manual trudging thru. Thanks for the comments guys. Yes, this stuff really isn't hard - although you'll get lots of gfx guru's trying to tell you otherwise. In fact as you will learn in life.. most things arent really very hard at all once you have the tools and knowledge :) Has been 30+ yrs of coding for me, and last 10-15 being media/games/sims based. I feel its only fair to pass on some little 'bits of info' :) I'll try and get my webby up again too, theres a tun of other junk I have done previously that people might find useful/interesting/funny/poinltess :) Hopefully be back on the weekend with some 'stuff'. |
| ||
| The world of programming is full of that kind of phenomenon. People appear to pretend passionately or at least compulsory that their accomplishments are the result of some kind of alchemy, academical or otherwise, haha. I remember when I had a look at siggraph papers about things I had to deal with myself and solved already. They wrap their discoveries in a shroud of mystery that only universities could appreciate, declaring those outside their indoctrinations as inadequate or even dumb at times. They elevate the vibrational kinetics of dihydrogen monoxide to raise the dynamic equilibrium of it to the verge of shifting from liquid to gaseous state by inducing transformed energy over a temporal period instead of cooking water. Or something like that... Eitherway, sometimes it's really not even malicious, but simply the fact that they've been brainwashed to regurgate their indocrinations, which went beyond their understanding right from the start. They simply don't know any better. That's why I don't ever like to just use someone's routines or formulas, but need to understand what I want to do exactly. Once you do, you can skip over plenty of accumulated bullshit and get yourself and the machine right to the chase! I must say, though, I like GLSL, even if it's once removed from the beautiful assembler, it tends to be far more relaxing to deal with, of course. But I love the challenge... so I'll dive a bit deeper into your approach here, too! Thanks again and I'm excited about anything you'll present next! :o) AH, the only thing that would've made me even happier from the start, would've been a demo that simply draws a few quads without the need of any loaded geometries just to get started. If you'd want to make your tutorials even more approachable, you could consider something like that. I think I would've fallen in love with them immediately! |
| ||
 |
| ||
| Just read this whole topic, quite enjoyable learning curve, thanks grover. |
| ||
| Ok. Im back!!! :) Thanks Snixx.. wiked pic :) After knocking off a couple more sims, and now starting a couple of new projects I thought I might try and fit in some more into this tute. Lighting is high on my list, so I want to make sure these get in within the next week. Thats my aim, but my two teenagers, and life tends to get in the way. But I'll try to complete by the end of next week. Id also like to dicuss something thats been annoying me a bit, and thats use of a gui. I really need gui's for tutorials (do it all the time at work) so after this lighting update I will be adding a really neat way to do extremely professional looking gui's (flash based gui systems built within the 3D env). The way this will be done is through a flash 9-10 compatible object. Damn useful stuff :) Anyway.. back to lights. With lights, will come shadows. Lights wise we will implement global lighting, local lighting and the point lights I mentioned earlier. So, first things first. Global Lighting ------------------ Generally global lighting can be extremely simple right up to utterly complex (with all sorts of light scattering algorithms). Our implementation will be somewhere in between: - General scattering routine (so nice sunsets, night, and distance fogging) - TOD (time of day) integration. - Generic global shadows (simple projected textures - with flaggable objects that can be set as shadow targets and shadow casters). Global lighting is a fairly 'early' pass in the defferred renderer - this is mainly because things like bloom and other post processes tend to be applied afterwards. Our global lighting will be a sort of shadow and colour mask, that is blended with our plain 'albedo' pass. SSAO would be added after the global illumination. So.. now I'll add the code, post some screenies and the source. Back with some GI soon... |
| ||
| Hi Grover, I thought this whole thread was rated as "awesomeness of epic proportions". Then I saw this: ...(flash based gui systems built within the 3D env). The way this will be done is through a flash 9-10 compatible object... This brings me to a point where words can no longer describe the status. To me it sounds like you are trying to teach us how to make a fire by setting up the sticks right, rocks around them, then you hand us a flamethrower. :o) This has got to be the best thread I've ever read and been a part of. *opens eyes wide* *watches Grover code magic* |
| ||
Ok _JIM.. heres a sample of the flamethrower... Im cleaning up code a little and making things easy to understand, but I'll detail this in the next few days..  I should point out, there are two flash movies (entirely separate) running in this sample - the clock/calendar movie on top left, and the simple HUD on bottom left. All gadgets and such can interact directly with the scene, and vice versa. Hopefully people will like this (it will make adding the lights and such easier - need nice controls to use it). Oh. Point of note. The code is fairly heavily Win32 - uses another DLL. However, this is not to say it cannot be converted to a linux .so or something similar - the interface is quite simple. Will publish the DLL and bmx's soon. need some sleep.. wife is wondering just what the hell Im doing up at 2:30am.. :) |
| ||
| Wow amazing! Can't wait to see it in action :) |
| ||
| Ok. Now you can play :) http://users.on.net/~dlannan/Files/gldemo_06.zip Some points of note: 1. I have not tested much of the movie interface, it _should_ work fine (Play, Stop, GotoFrame, Hide, Show) but if there are bugs its because I decided to get this up now. I will test later this weekend. 2. Callbacks / Bound functions Be a bit careful with these functions they can be a bit tricky. I currently only support a single string argument, so dont try calling anything more!! This will support multiple args but later.. when I get some time.. :) 3. No dll source yet. The source is based on the LGPL code from the Hikari project: http://code.google.com/p/hikari-library/ I will post this once its a little more stable/clean. Flash side: To call functions bound with the flash movie use the "ExternalInterface.call()" capability. This is Flash9/10 compatible. However dont let this annoy you if you want to use earlier flash versions. You can build Flash 4.0 movies but will be limited to only calling into flash and not being able to call out from Flash. Heres an example call used in the controls.swf movie: ExternalInterface.call("exitClick"); Can you guess what it does? :) Take a good look at how the Hikari library is initialised, and how controls/movies are loaded. Examine the dll interface for some fairly obvious function calls, but feel free to contact me if you have any issues. Following this, the lighting additions will appear (I wanted a nice UI for the sun control and the scattering controls). Interestingly too, if people are wanting to use this dll in a normal OpenGL application you can, using the same interfaces. I have decided to also turn this demo into a small game.. so the tutorial will eventually be a little deferred renderer game that people can pull apart and build their own gear with :) Finally if anyone is interested, I finally managed to get some of my other 'stuff' up on a website: http://www.gagagames.com. I will get this tutorial posted here and organise it a little more into something a bit easier to follow. |
| ||
I thought Id add this.. I did a quick test to try out some modern made movies - swf's. So I added a little carousel made with papervision 3d (for flash) and heres the result: To be honest I thought there may be issues, but it worked quite flawlessly. So if you want some snazzy GUI's with nice 3D.. there you go. I took the movie from this website: http://clockmaker.jp/labs/ It has some utterly fantastic demo's in flash. Well worth a look just for the visuals :) |
| ||
| Sweet! I thought you might be using the Hikari library. :) Sadly, Win32 only is somewhat a showstopper. However it does look awesome! |
| ||
| There is another way.. its a little.. erm.. different.. but would depend heavily on a html widget/renderer. Basically there is this: http://github.com/blog/579-flash-in-javascript However this method is utterly limited (doesnt support many of the great features of flash) and really is so much work its easier to just bind to the driver for the appropriate system. It is quite easy to bind to the Linux/OSX flash object (yes there is one for each of these platforms :) ). When I post the code, you can see. You replace the COMMs interface with a different driver (they all have similar setups), and you have the same results. As I noted above, I wont post Linux/OSX/Non NVidia solves because the amount of extra hassle needed to do so. This tutorial is more about the architectural level than the implementation. I should point out too.. thanks to dumbass Apple and their "non Flash" attitude to iPhone.. they are missing a massive audience they could add to their browsing capabilities. < personal stab at Apple - I really hate them, even more than Nintendo and MS these days > Finally. When it comes to UI's. There just isnt a better way to do this. The tools for flash are excellent and allow artists to be completely creative with menus and such (why EA and THQ use similar systems heavily) and it makes for _simplistic_ GUI implementation in code. There currently isnt a better middle ground at the moment. For comparison, JavaFX still requires a huge amount of code binding, thus a mass of state mechanics and such, and very little decent authoring tools for the artist, HTML is just too messy (with/without CSS & JS) and very inconsistent on different platforms, and Silverlight is a joke. Code based solutions are generally just unable to compete with development time comparisons. A basic example, in a recent sim I did, I added a Flash interface with an artists assistance in less than 8 hours - fully operational, with a number of screens and buttons and controls. The same interface we traditionally used to do in Java+Swing+SVG took an average of over 1 month!!!! thats code.. and pixel placed art.. OK.. onto lighting.. GI.. and Scattering.. here soon.. :) hopefully in next couple days.. |
| ||
| Nothing new? :) |
| ||
| So sorry.. back on holidays.. and will be updating here too :) Also.. will be over at LE too.. :) |
| ||
| This thread. |
   |