Leadwerks Engine BlitzxMax module - test 2

BlitzMax Forums/BlitzMax Programming/Leadwerks Engine BlitzxMax module - test 2

JoshK(Posted 2007) [#1]
Here is the second BlitzMax module test. The engine has been completely rewritten over the last two weeks.

http://www.leadwerks.com/ccount/click.php?id=44 (5.43 mb)

Features include:
-Super-fast rendering with mesh instancing can handle 1,000,000+ polygons.
-Blitz3D-like command set, including mesh and surface contruction commands.
-Dynamic lights, projectors, and shadows.
-Z-sorting for transparent objects.
-Fast terrain decorations for grass, rocks, etc., can handle tens of thousands of instances.
-Complete command set for rigid bodies, joints, and ropes.
-Easy-to-use command set for creating vehicles with arbitrary number of wheels.
-Load DDS compressed textures (more than ten times faster than loading textures from image formats).
-Store media in package files (.zip or .pk3).
-Extendable particle emitter entity.
-Built-in corona entity for light halos.
-Extendable mesh loader type to add support for more file formats.








degac(Posted 2007) [#2]
downloading now!!!

[edit]
Marvellous!

Can you explain 'better' how culling works in your examples? Sometime 'pieces' of world disappears without a logic reason. There's culling - portal zone hidden in the world?

Great job...any ETA for the final version?


Tachyon(Posted 2007) [#3]
Looking great. Very exciting stuff going on here!


JoshK(Posted 2007) [#4]
There's currently a bug whereby particle emitters get overculled. Did you notice anything else?

Culling works by frustum culling, occluders, and portalzones. Occluders and Portalzones are not demonstrated in this package, but I will have more info on them later.

Portal and occlusion culling honestly aren't that important, because under most circumstances it is fastest just to chuck everything in the frustum at the graphics card, especially on newer cards. You have to use portals carefully, because they will only make a big difference in certain conditions.

This is pretty near-final. It's definitely usable, and there is a lot more functionality I didn't take the time to make examples for. You can write a lightmapper, for example, or add your own a skeletal animation system, or write a .b3d loader. If it's in the documentation, it works.

Of course I have more features I intend to add like shaders and a built-in animation system, but there's no reason you can't get started now. The demo module times out after 60 seconds, so just order Leadwerks Engine from my site and you'll get the unrestricted module.


mongia2(Posted 2007) [#5]
thanks!!!!


(tu) sinu(Posted 2007) [#6]
Looks great, does the blitz like commands work exactly the same as blitz3d ie turnentity, rotateentity, deltayaw, tformvector etc?
Those commands are priceless to me and i would definitely buy this if they work the same as in blitz3d.


JoshK(Posted 2007) [#7]
Yep, I originally wrote the positioning commands by trying to duplicate the results Blitz3D gave. That was probably the hardest part of this whole process. There is currently a problem with scaling parented entities...the results might not come out right if you scale an entity and then parent it. But other than that, everything works perfectly.

The engine uses quaternion rotations, so you can smoothly interpolate rotations. Animation is not presently supported, but when I do add it, animations will appear correctly because it is using quaternions.

Take a look at this demo. It shows some pretty complex transformations in action:
www.leadwerks.com/post/quat.zip


LarsG(Posted 2007) [#8]
is this going cross-platform, Leadwerks?


mongia2(Posted 2007) [#9]
in you demo test 1

i write entitycolor mesh,255,0,255,0

but mesh is a grey!

thanks for addition quad!


JoshK(Posted 2007) [#10]
My engine relies more on the materials system rather than the Blitz3D surface/brush convention. EntityColor only has an effect on certain entities like lights or projectors. Because objects are rendered in batches (and in the case of bspbrushes, collapsed) it doesn't make sense to allow a color per object. Coloring objects would be a simple way to control visuals, but it doesn't jive well with the lighting I have implemented, and even less so with shaders.

Likewise, you only use vertex colors in my engine for static mesh lighting. The Blitz3D approach was almost a wrapper for DirectX settings, and it didn't handle things like lighting very well. I've taken steps so that you don't have to write your own lighting and other systems, but it may require you to get used to a slightly different approach than Blitz3D.


Knotz(Posted 2007) [#11]
Does it load b3d files, or have I to write my own loader?


JoshK(Posted 2007) [#12]
Doesn't load .b3d files ATM. I think I have some old Blitz3D code to manually load .b3d files, so if I can find it I will plug it in.


(tu) sinu(Posted 2007) [#13]
Sounds better all the time, i hope the speed of the animations can be changed while they're running, unlike blitz where you have to do them all with updateworld and not each individual animation.


AdrianT(Posted 2007) [#14]
Hey, this is pretty good :), you've come a long way in the last couple of years.


Chroma(Posted 2007) [#15]
Looks good.


JoshK(Posted 2007) [#16]
I created a .b3d mesh loader. It doesn't handle materials yet, but this shows how you can add support for additional formats. Most of this is Mark's original .b3d parser code. Just put this in the engine directory and run it. Change the name of the .b3d file to load, since I didn't include the jeep b3d in the package.



JoshK(Posted 2007) [#17]
I just spent a lot of time going back and forth between 3ds max, Blitz3D, and my engine. I forgot that .b3d files use a nonstandard quaternion. My rotation code was working fine, but I had to convert the b3d quaternions to standard ones.


degac(Posted 2007) [#18]
Silly questions:
1. Is there somewhere a help/list of command supported? I've downloaded the SDK, but some commands are non recognized - I though there were 'differences' between the two product (SDK and Engine Module), but I tried. [edit]After crawling in you forum I found this http://www.leadwerks.com/files/ProgrammingGuide.chm I suppose THIS is the right help guide, right? But I think it still miss something...
2. I see the game supports shadows (shadow map I suppose) for the moving-object (car, crate & so on...). In future will there be other shadow type (as volume shadow) or the implementation is left to the programmer?

Thanks


CodeGit(Posted 2007) [#19]
I really like the way this is heading. Nice :)


Dreamora(Posted 2007) [#20]
shadow drop sadly still bugged.

on the sample scene shown above if you go with the buggy right to the walls on 2 of 4 of them it will cast a quadratic shadow echo. No idea wherefrom this comes but if you are driving next to the wall it will show up.


JoshK(Posted 2007) [#21]
The docs are in a state of flux right now. Most commands work just like B3D. The old .chm file is documentation for the scripted engine, and is outdated. The commands are listed in the BlitzMax help docs, but very few are documented.

I'm still doing some thing to improve the projected shadows.

I don't have any definite plans for additional shadow techniques. I chose projected shadows because they are pretty fast, look good, and can be used in an actual shipping product. There are some other techniques, but they significantly limit the size of the world you can display.


Ian Thompson(Posted 2007) [#22]
I quite like the current projected shadows, my only gripe is that some see to be getting projected from above an object... but for from speed to quality ratio... there more than acceptable..


JoshK(Posted 2007) [#23]
The shadow issue is being addressed. I could do a quick fix, but it would slow down performance. I'm talking to Julio about a way to eliminate those faces from the calculated BSP.

I also changed it so a shadow will render all a mesh's children. Now I have a single shadow for the whole car, instead of separate shadows for each limb. (That was why the wheels looked darker when they overlapped the chassis shadow.) This will work the same for animated characters.


*(Posted 2007) [#24]
Looking amazing, especially like the vehicle test :)

Doesnt run on my laptop but im not amazed by that


Dreamora(Posted 2007) [#25]
good news :-)


JoshK(Posted 2007) [#26]
Added decolayer animation, so the grass sways. Also added some new sprite view modes that will just rotate the sprite around one axis to face the camera. This can be used for tesslas, lighting, or as shown here, headlight beams:



Matty(Posted 2007) [#27]
What I could see looked fine, but Example06, Example07 and Example08 (executables) loaded, displayed 1 frame and then exited so I don't know what is in there. I don't have a copy of BlitzMax so can't test the .bmx files.

My laptop specs are:

Asus A4K, AMD64 3500+ (2Ghz), 512 MB System RAM, ATI Mobility Radeon 9700 64MB VRAM, WinXP Pro

Am I doing something wrong?


degac(Posted 2007) [#28]
Hi, I've tested on a portable (Pentium M 2.13Ghz, 1gb ram, WinXp pro, Ati Mobility FireGL V3200, Directx 9.0c) but I've problem running (and compiling) example 6,7,8.
Any clue about the possibile problem?
(The mesh instancing runs at the incredible speed <1FPS!, but it runs!)


SebHoll(Posted 2007) [#29]
All examples run fine on my PC (Intel Core 2 Duo, 2GB RAM, Vista Ultimate, nVidia 7600GT) however, I noticed that in the car samples, when driving under the ramp the shadow appears on top of the ramp :-S

Really nice module though! Well-done!


KimoTech(Posted 2007) [#30]
Really nice project u got there! Is it shadowmapping or soft-stencilshadows u use there? :D


jkrankie(Posted 2007) [#31]
This new version runs better than the last one for me, but still not perfectly. The demos 6,7 and 8 don't work at all (they flash up a frame for a window then exit before anything is drawn in the middle), and the other demos look a bit like this:

www.charliesgames.com/ledpic.jpg

It's running on an Intel GMA chip BTW.

I hope you can get compatibility improved, because i really like the module.

Cheers
Charlie


JoshK(Posted 2007) [#32]
I'll get it figured out.

The shadows are projected textures, like what HL2 and Unreal 3 use.


ImaginaryHuman(Posted 2007) [#33]
What version of OpenGL does your engine need, and what extensions?


JoshK(Posted 2007) [#34]
It requires 3 texture units and the VBO extension. I can probably do without the VBO requirement, but if a card doesn't support VBO it almost definitely doesn't allow 3 tex units.

3 tex units are required for proper rendering of terrain. Each layer has a separate texture, lightmap, and alpha map. My artists have been begging me for months to allow a separate lightmap and alphamap, and in a situation where backwards compatibility holds up progress, I think it is best to move on.


JoshK(Posted 2007) [#35]
I set up an animation keyframe system. It uses a binary search to determine the two closest keyframes to any time value between 0.0 and 1.0, and interpolates them. I also added 3 loop modes that can be used to loop or extrapolate the animation.

You create an animation sequence:
animseq:TAnimSeq=CreateAnimSeq(frames=0)

Then add keyframes:
AddKeyFrame animseq,frame,x#,y#,z#,pitch#,yaw#,roll#

Then you call Animate:
Animate entity:TEntity,animseq:TAnimSeq,time#,loopmode=0,rate#=1.0

Animate does not do any continual updating at this point. It just interpolates a keyframe for time# and orients the entity to match that position and rotation. The rate# value will interpolate between the current orientation and the specified one, so you can use this to blend animations together:
Animate entity , runseq, time#, 0, 1.0 'set the entity orientation to the exact "run" sequence
Animate entity , lookseq, time#, 0, 0.5 'set the entity orientation halfway between the current orientation and the "look" sequence.

I think this method requires a little more work than just animating the entity once and letting the engine update it, but it gives you a lot of direct control over how the animation works. Also, once you start doing anything interesting, you'll find you need to continually switch animations anyways, so you might as well do it from the start. Finally, this avoids the need to attach an animation to an entity in any way, so that means less spaghetti for me to sift through.

I always had difficulty with Blitz3D's animation system, and I think pure stop animation with a lot of control might be the way to go.


JoshK(Posted 2007) [#36]
Here's a demo:
http://www.leadwerks.com/post/anim.zip

I was a little worried at first, because the animation (especially skin weighting) is pretty CPU-intensive, but I am getting 1000 FPS now, so hopefully it is fast enough it can display enough actors without choking.




mongia2(Posted 2007) [#37]
thanks for animation

in my test is:

bmax 500-450 frames!

blitz3d 500-498 frames!


JoshK(Posted 2007) [#38]
Blitz3D seems slightly faster. I'm not sure why, unless it's using some DirectX thing that transforms the vertices on the graphics card.

I tried some cyclic updating stuff that only updates a set number of verts each frame, and gained a 33% increase in speed that way, with no noticable glitches. However, I think the proper way to do this is to write a vertex shader that transforms the verts on the GPU. For now I will just use the CPU transformations, and we'll see about adding a faster method later.


Stu_ovine(Posted 2007) [#39]
Looking nice....

It is taking quite a long time to load the initial screen - but runs fine once its loaded.

Are you still planning shader support ?


JoshK(Posted 2007) [#40]
Yes, I have a good bumpmap/cubemap shader written, and shaders are actually implemented right now (CreateShader(), ShaderSource(), CompileShader()), but I want to get the rendering base down first.

I am adding support for .smd files right now. It's the uncompiled HL2 model format, and it is very widely supported, and I think it makes animation very easy.


JoshK(Posted 2007) [#41]
Here's a full-res HL2 model. It takes a while to load because it is optimizing the mesh and smoothing the loop, but it works:
http://www.leadwerks.com/post/anim2.zip

The engine will allow up to 4 bone attachments per vertex. I added a function that will loop an animation, and used it to fix some jerkiness in this animation's loop. I may even write a new vertex-weighting prog like my old "Character Shop".

I'm pretty happy with the speed, because you normally would use low-res LOD models for further away characters. The engine runs so fast with everything else, and I figure characters are going to be one bottleneck where I spend all those saved frames. My guess is this would run at about 80-120 FPS if I loaded a complete scene behind it.




z4g0(Posted 2007) [#42]
WOW!
the framerate il quite hight, but the animation look a bit 'fragmented' (seems there's no interpolation, showing only the keyframes, or something like it).

For the .b3D importer, there are some updates? I'm really curious to see some .b3d static mesh with materials in your engine!

Thx!


degac(Posted 2007) [#43]
It runs very smooth.
@Leadwerks: can you create a example with 'many' HL2 model running at the same time to see how it works?


smilertoo(Posted 2007) [#44]
Thats looking very good so far. Any chance of a fast 3d skinned format other than .b3d? (its a pain to make them unless you have 3dsmax).


JoshK(Posted 2007) [#45]
That's what .smd is for. It's an ideal format for skinned meshes.


smilertoo(Posted 2007) [#46]
your smd example is 10x slower than the b3d sample on my system.


JoshK(Posted 2007) [#47]
That's because it's processing ten times as many vertices.

To handle a load like this, it needs to be done with a vertex shader. Older games that performed the weighting on the CPU used much lower polycounts.


(tu) sinu(Posted 2007) [#48]
good idea with the smd file choice. There's a milliion tutorials, tools and convertors for that format and models people can use as placeholders etc.


smilertoo(Posted 2007) [#49]
cant opengl access hardware skinning?


JoshK(Posted 2007) [#50]
cant opengl access hardware skinning?

Yes, it can.


mongia2(Posted 2007) [#51]
please write a manipolation texture

es:
blitz 3d
texture buffer()!!

thanks!


JoshK(Posted 2007) [#52]
You can do this:

pixmap:TPixmap=LockTexture(texture:TTexture)
'Draw stuff on the pixmap here
UnlockTexture()'updates the texture

I also have some FBO stuff implemented internally, and I can use this to render a camera to a texture. This is useful for post-processing effects like bloom and other full-screen shaders.


mongia2(Posted 2007) [#53]
thanks!!

i use for point_entity:

for example camera

Function Point_Entity_camera(entity:tcamera,x#,y#,z#)
Local xdiff# = EntityX(entity)-x#
Local ydiff# = EntityY(entity)-y#
Local zdiff# = EntityZ(entity)-z#
Local dista#=Sqr((xdiff#*xdiff#)+(zdiff#*zdiff#))

Local pitch# = ATan2(ydiff#,dista#)
Local yaw# = ATan2(xdiff#,-zdiff#)
RotateEntity entity,pitch#,yaw#,0
End Function

add in your engine!!!


siread(Posted 2007) [#54]
Just tried the sample demos but I get "Unhandled Exception: Unhandled Memory Exception error" when it hits RenderWorld. This is on my laptop with it's crappy SiS m760gx onboard card. I'll give it a try on my main PC. :)


ninjarat(Posted 2007) [#55]
Ok, most of them didn't work, but I've got an extremely old graphics card, NVidia GeForce2 MX/MX 400. I'll try again once I've upgraded.