Really large 3D scenes revisited
Blitz3D Forums/Blitz3D Beginners Area/Really large 3D scenes revisited
| ||
| I'm still getting sportatic ability to load correctly exported .x files, and larger scenes exported as .b3d files simply return Memory Violations and then crash badly (double crash with c lib exit crash) None of the objects in the scenes are anywhere NEAR 65K verts. (largest is 11K verts) About 100 objects in the file, a few have as many as 5 brush multimaterials, with each brush having only 1 texture map in it. Tell me I don't have to split up this scene into its myrad components to make it work. |
| ||
| Can you load these files in a non-Blitz tool effectively? Attempting to do this will tell you if the problem lies with Blitz3D or with the file. If practical, upload some of the models that are crashing so we can take a look at what's going on with them internally. Also, what version of Blitz3D are you running? Do you have the latest version available from your account page? |
| ||
| Thank you for responding! It's a 5 floor video arcade. Answers: 1. Yes 2. I'd love to. 13 megabytes. How? 3. The latest one I got the scene to load by splitting the scene into 3 parts: 1. The main scene (walls, floors, benches, trash cans...) 2. The >collapsed< arcade games object split into 2 files each with 4 parts of 11K vertexes each PRIMARY PROBLEM NOW: Sporatic artifacts noted where new objects cause parts of the scene to disappear and to no longer be rendered. |
| ||
| You really need to post some screenshots at least for some of these problems you're having. It's hard to understand the specific problem from a short text description. Artifacts from items that are too close together may be caused by z-fighting. This basically occurs when two or more polygons are too close to each other and the graphics card tries to render both in the same space (sort of). Try moving the items away from each other slightly. It may be a different cause, but it's hard to say with the info available. For hosting files, try: http://www.mediafire.com/ For hosting images, try: http://imgur.com/ |
| ||
| To be honest it still sounds like the vertex limit problem to me. 1) How are you counting verts? There may be a different number in the exported model than are registered by the editor program (unwelded vertices aren't very useful to editors, where you normally want to manipulate whole surfaces) 2) 65536 isn't a particular B3D limit, it's a common graphics card limit. Half that number is also moderately common; some cards on mobile devices (still manufactured in 2012) can go as low as 2048! If you have 11K vertices in the editor, and it exports all meshes unwelded, you could well be going above a limit of say 32K. If it is the vertex limit, you don't need to split the file up. Just to split the largest surfaces up. Also you should make sure that welded verts are being used wherever it's appropriate to do so, simply because if your scenes are that large you might get quite a memory saving out of it. |
| ||
| Max 5 objects are welded by nature unless you explicitly split the relevant polygons/surfaces up. I had tried splitting the largest object up into smaller parts in the same file and the file refused to load. The only way I could get the scene to load was to split it into smaller files even though none of the objects exceeded 11K vertexes. I do not use scenes with exploded tris any more. I will try to include a pair of screen shots to illustrate the problem. The 1st image represents the scene at the threshold of the issue, the 2nd shows the scene when I load a couple of more test models.   |
| ||
| Those are probably skeleton animated meshes. The way skeleton animation works is that bones drive the polygons around even if 'away' from the mesh's pivot point (which always remains static). Try selecting one of those characters and seeing where Max draws the 'transformation gizmo' (that set of 3 colored arrows used for moving objects). If it's not centered to them, you're gonna have problems. I assume Blitz3D arranges the mesh's culling box to encapsulate all polygons for just the first frame of skeleton animation, wherever these polygons might be. Along an animation sequence your mesh will move around and probably go away from the starting position where the cull-box is placed, so sometimes this culling box will go out of view and for all Blitz3D knows the mesh is not visible because of this, so it hides it. If those characters are 'walking around' using nothing but skeleton animation, eventually the polygons will be driven so far from their starting position (where the cull-box was created) that it will spontaneously disappear for no reason. The way to animate skeletal characters around is to have them "walk" in the same place by skeleton, but animate their pivot moving and turning so they actually move somewhere. It's a combination of these two things. This is how you make real-time cutscenes. If you move the pivot... - The mesh's cull box moves along with it instead of staying at the same place. - The bones move too regardless of whatever animation they might be performing. - The collision object also moves. So the whole entity moves. Last edited 2012 |
| ||
| Thank you! I will check the pivot points of the original max objects prior to export. They are presently un-animated. Anyone else have observations regarding memory consumption? Last edited 2012 |
| ||
| Tried recentering the relevant mesh's pivot points: NOPE. Clearly this is RELATED to some theoretic maximum number of polygons that the blitz will allow you to render in a scene. My observations (using vanishing polygons as shown as a gauge): ABSOLUTE MAXIMUM POLYGONS COUNT: 703425 83 actors (nothing else in the universe) 568039 67 actors skybox --- not even visible in the last scene - curious, only adds 214 380106 42 actors sb+bldg -- additional 23K for building Oh, and my problem was - it only allows 11 actors when I add the arcade games into the scene (visible in the 1st-left image above as black boxes hanging in test position to be visible in the frustrum all at once) They add about 140K more. If I try to load too much more, I'm then rewarded with Memory Access Violation on the RenderWorld statement Hey, um, what if I am using polygons that only have a material painted on them? I noticed the green leaves were only on 1 tree in the building scene. And you can see the unpainted benches also. Do I need to start using 1 pixel texture maps? http://sites.google.com/site/mikorians/ArcadeBldgGames1.zip Pentium 4 3.4ghz 1GB, Nvidia FX5500 256MB Last edited 2012 |
| ||
| I don't know about loading a large mesh but i successfully created and renderd a single terrain mesh with over 2 million triangles. I think the max triangles for a surface is 65535, just make sure no surfaces exeed that number. You don't have to split up meshes, just add new surfaces. Edit:This was in minib3d but it will probably work in blitz3d to. Last edited 2012 |
| ||
| Oh, and my problem was - it only allows 11 actors when I add the arcade games into the scene (visible in the 1st-left image above as black boxes hanging in test position to be visible in the frustrum all at once) They add about 140K more. Well the first thing is that this part is a bit ridiculous. At that distance from the camera, those arcade cabinets should be adding exactly 16 polygons to the scene because any scene with this kind of detail absolutely has to 1) have a LOD system to swap out the high-detail mesh for a simple one with a simple texture, at ranges where the details aren't visible, and 2) identical objects should be sharing mesh data: the number of cabinets in the scene should make no difference because you absolutely have to be CopyEntity-ing them rather than just adding more surfaces. The argument that they ought to be there for the sake of testing is a bit meaningless: this is not how one ought to be displaying a scene. Scenes are built by LOD-ing and sharing surface instances. DirectX 7 isn't advanced enough to do this automagically, as DirectX 11 would, so you have to write it yourself, but that's all. So investigating a more efficient scene layout that doesn't duplicate assets, and a LOD system to reduce the quality of those assets as far as possible, is probably a priority. The same by the way goes for the animated actors: share surface data with CopyEntity and you should be able to have at least 20 on screen without multiplying the effect of the polygons (eventually the animation time will add up, being CPU-driven; that varies depending on the animations' complexity but is a separate issue). If you're already doing this then sorry for the irrelevant commentary, but it sounds very much like you're not. Even modern AAA game engines can't work with un-instanced entity data (in fact, the multitide of ways of cheating are how they achieve nice HD). Secondly... what's the surface breakdown of your scene? We really need more information on this, because as has been mentioned a few times now, the surface is the basic unit if rendering. Whether you're sharing surfaces, and how large each one is, is far more important than the cumulative mass of all surfaces in the scene. If your scene is not commercially-sensitive (though judging by the level of detail, I assume it is), perhaps you can upload it for the rest of us to examine? If it is... can you perhaps give a full statistical breakdown of the scene graph including surfaces, vertex counts, instances, etc. all in their proper hierarchical places? |
| ||
| The link to one of the problem sub components is above. It loaded fine without all this hassle in my previous dx8 engine. They're all duplicates because it was simplest to convert a few models. I assumed it was just load-and-go. It's just a bunch-o-boxes and actors. I know that $50 for an engine is chicken feed. I wanted to create MORE ARTISTIC scenes to use in my movie. This scene was the largest in my dumb old engine, and I have to test with SOMETHING. I know it isn't the most efficient, but I was happy using b3d until... Sorry if I upset anybody... geez... If I'm abusing b3d, I wanted to know how. Thanks for responding. I think personally that b3d is really really fast. But I also need accuracy. I think that if an object's in the frustrum and not behind anybody else, and not too far from the camera, it shouldn't be culled, right? And--- it shouldn't be culled FOREVER, right? They never come back. Last edited 2012 |
| ||
| I agree with Yasha. You should consider to use a low lod and a high lod for each mesh which has many details (tris) and to use CopyEntity(Mesh) to represent the same mesh when possible. Also for characters, you should consider to use one low lod mesh, with less bones, to represent all characters who are far from the camera. Last edited 2012 |
| ||
| I'm just trying to determine WHY the objects are partially culled, and/or removed forever from the scene. They're not animated and their pivots are centered. I don't know how to use LOD or progressive meshes in b3d. I do understand, but want to try something fancy when I understand the basic. The title of this topic is really large scenes. I have more actors than just the one, so duplication isn't what I mean. I would love to implement lod and low poly substitutes - please stand by. I like to test the limits of an engine before implementation of optimization. As I add more load, more seems to be removed in an unpredictable mannor. This troublesome scene. I guess the memory access error should be an indication that I shouldn't even try to load the whole scene, much less split it up and load the pieces of it. Last edited 2012 |
| ||
| Have you tried to export only a part of the scene (for example the stairs) and to check if the number of surfaces, vertices per surface, triangles per surface is the same when loaded in Blitz3d than when it is in you modeling app? I ask you this because when i was using the previous version of Fragmotion, the b3d exporter created duplicated vertices for each vertex and this resulted in a huge amount of total vertices in Blitz3d. (The current b3d exporter of Fragmotion works correctly) So maybe the exporter is corrupted ? Try it to be sure it works properly. |
| ||
| I cannot read a b3d file. One of the pieces is in the .zip file above. The max5 exporter is from www.onigirl.com/pipeline The viewer doesn't work on my system for some reason. Last edited 2012 |
| ||
| Sorry if I upset anybody... geez... Not at all. If anyone here's upset about anything it's that we haven't been able to help you with a problem that... seems trivial and somehow isn't. Respectively, for my part I'm sorry if I come across as overly abrupt (you'll know when I get annoyed: I'll abandon you to your own devices!). OK if you don't know how to implement LOD yet then don't worry about that part. RemiD's suggestion to check that the exported version actually matches what you were doing in the editor is what I meant in post #5, by the way: it is known that some programs simply don't do a very good job on this front. You can check this easily by simply calling CountVertices and seeing if it matches your editor's figure. I think that if an object's in the frustrum and not behind anybody else, and not too far from the camera, it shouldn't be culled, right? And--- it shouldn't be culled FOREVER, right? They never come back. Indeed, both are right. It's not going to be camera culling causing something like this (all that does it remove items from the render list - it's recomputed each frame and doesn't affect scene composition in any way). Unless of course its cullbox is messed up - that should only be a problem for animated meshes though, if they move out of the cullbox they were initially assigned. (OK, to get on with actually looking at the scene since it was provided all along...) |
| ||
| Thank you for understanding, Yasha. I have no external viewer that works. If you're able to load that object from the .zip file, then you're doing better than I am. Blitz wouldn't load it here. Last edited 2012 |
| ||
| The editors I have installed won't load it properly either (I can get a bunch of cabinets but nothing else). One idea comes to mind - can you export the model as .3ds from Max? .3ds is in general a terrible format - it will screw up your lighting and doesn't support bone animations, nor does it match up with the B3D entity/mesh system properly - but it has one really big advantage: rock-solid exporters because it's been around so long. If you can export it as .3ds (especially from an Autodesk product, since they invented it), it will guarantee that there's no problem in the exporter generating squillions of extra vertices or anything. For a static mesh and testing purposes, the downsides of the .3ds format should be irrelevant, so we can work from there. |
| ||
| You got the cabinets! OK, using countvertices, I tallied up all the vertexes for an ACTOR (4 surfaces) 8998, max shows the model to have 4708. I will now attempt consistency tests utilizing the .3ds format as suggested. I have written an .x file exporter macroscript for max. Should I attempt it using a paper I found on the b3d format? Last edited 2012 |
| ||
If we assume that the b3d importer of Fragmotion works properly, then this means that either your modeling skills are really bad or that your exporter produces a file with errors : I suggest that you try to modelize a simple mesh (i mean with only a few surfaces, vertices, tris) and see what tools you can use in order to export this mesh and to import it in Blitz3d with the exact same properties. If you can't do this, there is no point trying to fix Blitz3d when it is a file problem. Last edited 2012 |
| ||
| How are you guys getting from max5 to b3d then? Wouldn't want to reinvent the wheel... But I'm willing to! |
| ||
| I can't help for max5 to b3d. However i can suggest : Try to export your static meshes in .obj (you can save materials and UV parameters) Then add your pivots, influences, animations in Fragmotion or in Blitz3d or Find a better b3d exporter |
| ||
| I'm-a-gonna write one and post the maxscript right here! I just witnessed gile[s] b3d parser example and got more motivated! His example scene was more what I'm looking for! Last edited 2012 |
| ||
| Mikorians>>Here is a Blitz3d code i have used in order to analyze the b3d mesh exported by Fragmotion a few months ago. Edit : i have put in comments the lines which are not necessary to analyze a static mesh
Global GWidth% = 800
Global GHeight% = 600
GColors% = 32
GMode% = 2
Graphics3D(GWidth%,GHeight%,GColors%,GMode%)
SetBuffer(BackBuffer())
HidePointer()
SeedRnd(MilliSecs())
Global Camera = CreateCamera()
CameraRange(Camera,0.01,1000)
PositionEntity(Camera,0,0,0)
Origine = CreateCube()
ScaleEntity(Origine,0.01/2,0.01/2,0.01/2)
PositionEntity(Origine,0,0,0)
EntityColor(Origine,255,000,000)
EntityFX(Origine,1)
;HideEntity(Origine)
DLight = CreateLight(1)
LightColor(DLight,255,255,255)
PositionEntity(DLight,-1024,1024,-1024)
RotateEntity(DLight,45,-45,0)
AmbientLight(063,063,063)
CubeMesh = CreateCube()
ScaleMesh(CubeMesh,1.0/2,1.0/2,1.0/2)
PositionMesh(CubeMesh,0,1.0/2,0)
HideEntity(CubeMesh)
B3DMesh = LoadAnimMesh("test2.b3d")
ScaleEntity(B3DMesh,0.01,0.01,0.01)
Global SeqIdle = ExtractAnimSeq(B3DMesh,1,1)
Global SeqWalkForward = ExtractAnimSeq(B3DMesh,5,25)
JointXMesh = CreateMesh()
TPart = CreateSphere(8)
ScaleMesh(TPart,0.01/2,0.01/2,0.01/2)
AddMesh(TPart,JointXMesh)
FreeEntity(TPart)
TPart = CreateCube()
ScaleMesh(TPart,0.01/2,0.01/2,1.0/2)
PositionMesh(TPart,0,0,1.0/2)
AddMesh(TPart,JointXMesh)
FreeEntity(TPart)
HideEntity(JointXMesh)
XMesh = B3DMesh
SCount% = CountSurfaces(XMesh)
DebugLog("Mesh has "+SCount%+" surfaces.")
For SId% = 1 To SCount%
T_Surface = GetSurface(XMesh,SId%)
VCount% = CountVertices(T_Surface)
TCount% = CountTriangles(T_Surface)
DebugLog("Surface"+SId%+" has "+VCount%+" vertices.")
DebugLog("Surface"+SId%+" has "+TCount%+" triangles.")
VTotalCount% = VTotalCount% + VCount%
TTotalCount% = TTotalCount% + TCount%
Next
DebugLog("Mesh has "+VTotalCount%+" vertices.")
DebugLog("Mesh has "+TTotalCount%+" triangles.")
;SLipsId% = 3
;SEyesId% = 1
;SSkinId% = 2
;T_Surface = GetSurface(XMesh,SSkinId%)
;Red% = Rand(000,255)
;Green% = Rand(000,255)
;Blue% = Rand(000,255)
;Alpha# = 1.0
;T_Brush = CreateBrush()
;BrushColor(T_Brush,Red%,Green%,Blue%)
;BrushAlpha(T_Brush,Alpha#)
;PaintSurface(T_Surface,T_Brush)
;FreeBrush(T_Brush)
;JX = FindChild(XMesh,"Hips")
;DebugJX = CopyEntity(JointXMesh)
;PositionEntity(DebugJX,EntityX(JX,True),EntityY(JX,True),EntityZ(JX,True))
;RotateEntity(DebugJX,EntityPitch(JX,True),EntityYaw(JX,True),EntityRoll(JX,True))
;EntityParent(DebugJX,JX,True)
;HideEntity(JX)
PositionEntity(Camera,3,3,-3)
PointEntity(Camera,Origine)
Repeat
Cls()
If(KeyDown(17)>0)
MoveEntity(Camera,0,0,0.01)
EndIf
If(KeyDown(31)>0)
MoveEntity(Camera,0,0,-0.01)
EndIf
If(KeyDown(44)>0)
TurnEntity(B3DMesh,0,1,0)
EndIf
If(KeyDown(45)>0)
TurnEntity(B3DMesh,0,-1,0)
EndIf
If(KeyDown(46)>0)
TurnEntity(B3DMesh,-1,0,0)
EndIf
If(KeyDown(47)>0)
TurnEntity(B3DMesh,1,0,0)
EndIf
If(KeyDown(2)>0)
Wireframe(True)
Else
Wireframe(False)
EndIf
;If( AnimSeq(B3DMesh) <> SeqWalkForward)
; Animate(B3DMesh,1,0.3,SeqWalkForward)
;ElseIf( AnimSeq(B3DMesh) = SeqWalkForward And AnimTime(B3DMesh) = 25)
; Animate(B3DMesh,1,0.3,SeqWalkForward)
;EndIf
UpdateWorld()
RenderWorld()
Text(0,0,"Surfaces = "+SCount%)
Text(0,20,"Vertices = "+VTotalCount%)
Text(0,40,"Triangles = "+TTotalCount%)
Text(0,60,"Tris = "+TrisRendered())
Flip(True)
Until(KeyDown(1)>0)
End
You will need a similar thing. Last edited 2012 |
| ||
| Thanks. Well, I'm not able to make the export macroscript work. Apparently, the basic cube I'm trying to export has its verts and faces all messed up because of some b3d internal process I don't know about. It has 8 verts and 12 faces in max. Pretty straightforward stuff. Maybe I'm making a real stupid mistake somewhere. x,y,z nx,ny,nz, tu,tv (1 set of 2 as stipulated in the flags section) I have a scene_root node only around the mesh, do I need another node layer or something? Your program shows the twisted up cube with 9 verts, 16 triangles and 28 tris, and then when I close, we crash. Your program shows the b3dpipeline exported mesh happily with 24 verts, 12 triangles, and 24 tris. Both have 1 surface. http://sites.google.com/site/mikorians/test.zip Last edited 2012 |
| ||
| Shouldn't that be 36 verts? Does b3d optimize internally or something? I went through my test.b3d with a hex editor real carefully. It LOOKS alright... But not like the exporter generated test2.b3d. Whoops - saw the extra -1's not being in the exporter file. obviously wrong. Nope, still twisted up. We don't crash now though. It seems to be attaching one corner of a triangle to another without mercy... achk. (I tried 2 separate squares of 2 tris as one mesh) Last edited 2012 |
| ||
| I have put in comments the lines which are not necessary to analyze a static mesh. The program runs correctly but your 2 meshes have errors. A simple cube should have 1 surface, 8 vertices, 12 triangles Then if you add materials to some tris of the cube, some vertices will be added, this is normal, this is because the tris which have different materials but which share the same vertices need to have their own vertices. |
| ||
| To put it another way, a cube should have as many vertices as you need it to have. A Blitz3D cube needs 24 vertices because a vertex can only have one normal: for the sides to be lit with nice sharp edges and flat shading, that means each quad needs to be unwelded from the quads around it. A "simple" cube that doesn't care about lighting can share vertices between the surrounding faces. Either one is a perfectly valid B3D mesh, as Blitz3D permits, but does not require, welded vertices. A decent exporter should ideally be able to handle both types of cube without forcibly converting one to the other, so that you know exactly what's happening in your scene. |
| ||
I SOLVED IT FOLKS!!!! Here's a simple maxscript - I'm just gettin' warmed up! Yeehaw!!!global a global m global vrt i=geometry[1] fn writeblock a b = ( --Eliminates trailing 0 writestring a b fseek a ((ftell a)-1) #seek_set () ) clearlistener() --Write File a=fopen "D:\\Desktop\\test.b3d" "wb" writeblock a "BB3D"; flen=ftell a writelong a 0--White paper says the length of everything-AFTER-the-length writelong a 1 --Version Number=0.01 writeblock a "NODE"; npos=ftell a writelong a 0 #unsigned writestring a "Scene_Root" writefloat a 0 writefloat a 0 writefloat a 0 writefloat a 1 writefloat a 1 writefloat a 1 writefloat a 1 writefloat a 0 writefloat a 0 writefloat a 0 writeblock a "MESH"; mpos=ftell a writelong a 0 #unsigned writelong a -1 writeblock a "VRTS"; vpos=ftell a writelong a 0 writelong a 1 #unsigned --flags writelong a 2 #unsigned --num tuv sets writelong a 2 #unsigned --num coords per tuv (always 2) for t=1 to i.faces.count do ( v=getface i t for u=1 to 3 do ( vrt=in coordsys local (getvert i v[u]) writefloat a vrt[1] writefloat a vrt[3] writefloat a vrt[2] vert=getnormal i v[u] writefloat a ((vert[1] as float)) writefloat a ((vert[3] as float)) writefloat a ((vert[2] as float)) writefloat a (0 as float) writefloat a (0 as float) writefloat a (0 as float) writefloat a (0 as float) ) ) writeblock a "TRIS"; tpos=ftell a writelong a 0 --for t=1 to fc.count by 3 do ( writelong a -1 for t=1 to i.faces.count do ( f1=((((t-1)*3)+0) as integer)--You MUST use a calculation here f2=((((t-1)*3)+2) as integer)--because max WILL NOT produce f3=((((t-1)*3)+1) as integer)--the correct vertex numbers for the faces EVER. writelong a f1 #unsigned writelong a f2 #unsigned writelong a f3 #unsigned ) f=(ftell a) l1=(f-tpos)-4 fseek a tpos #seek_set writelong a l1 #unsigned l1=(tpos-vpos)-8 fseek a vpos #seek_set writelong a l1 #unsigned l1=(f-mpos)-4 fseek a mpos #seek_set writelong a l1 #unsigned l1=l1+(mpos-npos) fseek a npos #seek_set writelong a l1 #unsigned l1=l1+(npos-flen) fseek a flen #seek_set writelong a l1 #unsigned fclose a Now optimization X-file style is all we need. (I know how to do that!) Last edited 2012 |
| ||
| It's too bad the normals are tied to the vertexes (unlike x-files) because you can optimize the normal table separately... I like to have a switch for vertex normals or face normals (faceted vs. smoothed) |
| ||
| I think I fixed a problem like this a long time ago by just making everything tiny. I'd made a huge city out of the Driver demo I think it's called, and the further away from 0,0,0 I got, the more the wheels of the car would fall through the mesh. And that makes sense, cause if you're at say, 100000.0 z, you better not hit the gas, cause there IS no 100000.01 z in programming terms. You get eight digits, and that's it! However, with everything very small, I think you get more, but I don't remember for sure if it worked. I think it did. But try it. Try all kinds of crazy stuff, and keep posting updated versions for people here to work on. With the help of the people on this site, I was even able to antialias meshes. Last edited 2012 |
| ||
| Thanks for the encouragement! (???) By huge, I meant number of 'tris' We seem to bottom out at around 500K-750K -- I.E. The engine seems to start dumping vast quantities of tries/objects from the scene never to be rendered again. Solutions I now recommend: 1. Don't-use-that-many-tris... I just wondered why we didn't just get frame-rate degradation all to pieces instead. 2. The files won't load if they're too big, so bust 'em up into numbered scenes. 3. Use Fragmotion (I have it fortunately) to diagnose borderline scenes. 4. Use the pipeline to export despite the flaw noticed by RemiD because it's a nightmare to do it by hand (I'll keep poking away at it though - I can't resist). Just use the pipeline on the aforementioned scene fragments. 5. Why bother with b3d scenes? Because we can set up translucency and many other flags in advance so we don't have to hunt down the child objects and re-paint them later. This is what a file format is FOR. Last edited 2012 |
   |