Mesh "streaming" system
Blitz3D Forums/Blitz3D Programming/Mesh "streaming" system
| ||
| [edit] http://Danjeruz.servegame.com\DOwnloads\meshstreaming.zip [edit] I just thought id post about this.... Ive designed a lil system i think y'all might find useful. what it does is this: 1) Load mesh using Loadmesh command 2) Create a streamable mesh out of it using CreateStreamedMesh() 3) whenever an update is required, do UpdateMesh(streamedmesh,positionentity,distance) of course, this isnt designed to work realtime, every frame, but its good enough to run it every second or so. What it does is store the entire mesh in an array, and upon updating, recreates only the parts of the mesh within Distance of PositionEntity... Ive added this idea to Supernova and have managed to speed it up by almost 100% again by killing collision maths. (this is what i designed it for) Now, it has a couple of issues... like, Lightmapping doesnt work with it. but normal meshes seem to work fine. Of course, Lightmapping is quite widely used, so itd be great to see if someone fixes the issue. The other issue is that with very large meshes, theres a slight delay when the mesh is rebuilt. Thats why it should not be used totally realtime, but say, once a second, or when a player reaches the edge of the loaded section Well, what do you lot think of the idea? its pretty much complete, just sitting at home waiting for me to plop up here for y'all! |
| ||
| yeah this sounds pretty interesting... |
| ||
| sounds good if you can pull it off quickly enough... v. interesting idea anyway... i'd like to see the source, could be very usefull |
| ||
| What would you use this for? |
| ||
| By the time you have the mesh loaded, doesn't it make 'streaming' the mesh rather pointless? Just do multi-pass, take one instance of the mesh and set its properties as you need it and render every other instance of it with the same mesh (after setting the properties for those instances via a type or globals or banks or something). Just my take on this. Unless you're streaming the mesh all from the beginning, I just don't see a reason for you to have it in. So you can watch the polygons all get added on one by one? |
| ||
| Technically, "streaming" is the wrong word, i just came up with it so quick i wasnt thinking :P however, mesh load streaming i am planning to do also, a tad later on. Its useful for removing parts of a mesh that arent close to the player, for instance, so collisions arent checked for walls miles away. For instance, with Supernova, say i have 80 ships and a large level, each ship is collision checked with the level, and so is each of 500 bullets... the collision math can get quite intense... The level mesh is recreated around the player, and anything a certain distance away is not added to the mesh, so the mess's actual poly count can be ALOT less. say i only load 4000 polys of a 20000 poly mesh. big difference, large speed increase when using 500 bullets and 80 ships, because theres less to check collision with.. Anyway, theres a simple version here, which needs tweaking in several areas, its main two issues are it checks the average of each triangles vertice before loading, this means large triangles that reach from the player to a large distance might not get placed. the second main issue is that it can only do one set of UVs on a mesh, which messes up Lightmapped/multitextured models. All in all, this isnt the most impressive thing in the world at all, but it is/could be useful. http://Danjeruz.servegame.com\DOwnloads\meshstreaming.zip Ps, anyone want to host this, feel free, my servers started acting up again :( |
| ||
| I could see it being useful, if you could stream the textures in too, and you wanted to keep VRAM usage down, and you wanted large worlds. |
| ||
| well alot of these things i wouldnt be able to do in my time. Part of the reason im giving out the source! UNfortunately at the moment it stores each brush into an array. BTW i updated the zip. 1) it now works. (heh, forgot to add an include line) 2) It now checks triangles better (not 100% perfect) 3) It now supports both texture layers. (lightmapping compatible, etc) Woo, enjoy all. Source included. Post what you do with it :D |
| ||
| Nice, but if you're serious about this I suggest looking up oct_trees or a similar proven method.(I'm assuming you havn't.) Just to show how simple this makes the actual rendering, here's a recurvise oct_tree function, that can cull whole chunks of geo with simple frustum collision checking. |
| ||
| This is a waste of time. |
| ||
| how can you seriously suggest space partitioning is a waste of time dude? Along with lod I'd say it's one of the two most important core things an engine can have. |
| ||
| occlusion isn't quite the big deal it was 5 years ago, talking to people, a lot of commercial devs are going for more simple occlusion methods that can often be set up manually by the level designer/builders. As it often works out being more efficient than an automatic system, and often easier for the team bulding the game to deal with. Allthough, I don't know many PC devs, talking cross platform console stuff. Worked on a streaming game that was quite fun though, using 3dsmax Xrefs to define the rooms of a 3D baldurs gate type game, that streamed off of the DVD. Made PS2 development a lot easier, as you didn't have to reboot and wait for the game to load each time. Instead you just copied a new replacement room to the network, and in game walked out of a room, walk back in and the new updated room was allready there :) made things on the PS2 almost as easy as Xbox to troubleshoot. |
| ||
| Halo i dont know you but im quickly picking up on your arse of an ego. jeez, Youre saying nobody would find it useful? or do you just mean that proffesionals know better ways, and yes i ruddy KNOW pros will know better ways. bloody hell >.< go use your time making your games, its not as if theyre bad or anything. just stop wasting time poking other peoples ideas. If you came up with something as a semi-newb and you were very chuffed at it (i only just started playing with mesh alteration), to have someone tell you youre wasting your time? ahhh, well, thanks. Spacedman, im interested in this idea but it sounds a little advanced, i do need a better way of doing things though so if you could give a little more info id be greatful. cheers :D |
| ||
| Evak, that's the beauty of oct-trees, it creates cubic regions. It's not occlusion, it's quite literally deviding the map up into cubic regions, in a tree like hireachy. Is node 1 visible? Then all it's children nodes are. If not? None of it's children are. Oct-tree is a beautiful process...how it can take any abstract map and make sensible portals/zones. Occlusion isn't something I'd do for every chunk..what you would do is if a chunk peaks beyond a certain poly threashold, check if it's visible, even if it's bounding box is within frustum. |
| ||
| Are there any octree generating progies? my levels are already created and i have no modeling skill or anything. How would i implement it? |
| ||
| -EDIT- Actually, post an e-mail addy and I'll mail you the source. |
| ||
| now, for the latter idea, it would be as simple as naming parts of the level mesh and just loading the surfaces closest to the player, remoiving the farther ones, right? id do that if i knew more than how to place vertices and faces >.< il get there though, currently looking for a modeler again. As it stands though this system has inproved performance in supernova no end. Only downside realy is it needs fog to work and fog in space is surreal, so il be trying other techniques..... Cheers for that. Programmers will still find my program useful and easy to use though, depending of knowledge of course :) [edit] I missed the code Email: Damiensturdy@... [endit] CHeers |
| ||
| You should be grateful I choose to divulge the same thing you will learn for yourself after several weeks of testing. |
| ||
| just saying posting "this is a waste of time." is in itself a waste of time. |
| ||
| You know what they say: if you don't have anything nice to say, don't bother saying anything at all. Of course, I very rarely take that saying into consideration 'cause even if what I say isn't nice, I tend to include stuff that'll help the person I'm talking to. |
| ||
| Halo, are you refering to doing it b3d, or doing space partitioning in general? Like even in impleneted in the ground up.. I have it working, so I really can't see what you have against the technique. |
| ||
| there are things where oct tree is really needed. For example for the scene management to decide what you actually see and what not. To see a good implementation of it take Blitz3D. To see a bad one take DBP where stuff isn't hidden even when not within view frustrum. For realtime terrain, OctTrees are really usefull as well. But there are situation where they aren't needed: Shooter: Best use BSP like stuff with predefined partitions and a PVS algorithm to determine which partitions ( rooms ) have to be shown Ground only based games better use Quad tree instead of oct tree to save half the calculation time and nodes Space Shooters: might use a combination of OctTree and distance as they are normally very fast games and recalculation the visibility basing on OctTree every frame might be too much calculation for the amount of objects they have normally Implement an OctTree in Blitz3D is quite useless, at lest if it is used for occlusion stuff. You better split large levels into smaller ones ( might be usefull and needed for lightmap calculation anyway ), then they are handled by Blitzs occlusion without any probs |
| ||
| Don't build and break meshes, just hide and show them. |
| ||
| . |
| ||
| ahh, that is indeed a preferable way for a bigger speed increase... :) |
| ||
| It won't work however, as your average level is one mesh exported from a map editor. Oct-trees take a single mesh and split it up,which is the optimal method.(Well, bsp, and quadtrees and others are also an option) |
| ||
| Spaced man, thats what i was thinking more of, the octrees will give a speed increase, and will be hiding. "You better split large levels into smaller ones ( might be usefull and needed for lightmap calculation anyway ), then they are handled by Blitzs occlusion without any probs" problem is its not occluded from collisions. I dont care about other objects collisions in this matter, just the player ones. Enemies a certain distance away would be paused if their section of level isnt in view. As with my technique, anyone know if itd be quicker to move vertices rather than destroy/create them? |
| ||
| for interiors don't see why the level designers can't simply place floor triggers manually that turn room geometry on and off as needed. And make the smaller objects within a room mesh a child of the occluding mesh. you don't really need a clever solution, I think a primitive one can do the trick as easily and cheaper, plus save the coder a lot of time. Not so good for outdoors though eh? but this is what we were considering. As far as streaming meshes go, we were considering using a trigger system along with 3ds max's Xref system, which is supported within B3d pipeline. Perhaps Bmax will allow for fast loading/streaming direct from disk, something that b3d is terribly slow at. nice discussion , any system thats going to allow large chunks of geometry to be cut out of the rendering pipeline is worth looking into. If its automatic and doesn't use too much CPU I'm sure there would be a lot of people willing to buy it at sswift like prices, I know I would. |
| ||
| I use an octree in my engine. Even without vis calculation is speeds things up a lot, just having the geometry split into a big grid. |
| ||
| Um.. Now you confuse me Halo. Your other posts above give me the impression that you think octrees ain't worth the trouble, and then you say you are using octrees in your engine to speed things up. x) lol Unless you meant that octrees should be used only to cull complete meshes(hide/show entity). |
| ||
| Im too kind though, i know that when i first came to blitz, i had loads of trouble with too many polies and this kinda tool would have been excelent. I know also that others will look for something like it, so here they find it. Il prolly make a BC.com showcase or something. im also not a very efficient coder when im new at something, IM more hoping someone can do something with the code and share the results :) i have what i need from it now for myself, and il continue to add/redesign it. One of the things im noticing is that sometimes when you remake the mesh the creating of the vertices and triangles seems to take a huge amount of time. memory bottleneck or something? i dont know. What would be nice is if we all got together and created a good "realtime" occlusion system for blitz. Alot of us here have some great ideas! |
| ||
| Well I'll post a demo soon, but last night I rewrote vivid's oct-tree generation to be much more effiencient. And in a test scene I created, of a speed ball like areana, with brides lining it,and a huge monitor..all in all over 60,000 faces. With regular Vertex objects, no oct-trees, I pull around 100fps constantly. No matter where I look. With oct-trees, The fps runs at an average of 200fps. Even higher depending on where I look. Now, as I add to this level, the first method will get slower and slower..but as long as I use oct-trees, the fps is consistantly nearly 2x as fast. In short(Well too late for that), it's proving vital.. And the oct-nodes hug the mesh beautiful, however abstract and big/small it is.(Size is irrevelent.The oct-tree complexity stays the same) You can even set say, a 5000 poly theashold. if a area/node has more than that, it subdevides it further. so in lower poly areas there's far less oct-treeing..again making everything consistant and much much better than any hand placement job could hope to be.(and it only takes 5 seconds to do a 50,000 poly level on a p800.) |
| ||
| look forward to the demo, I hope you have some complex brushes/texture effects etc as 60,000 isn't that much when you have a low end Geforce FX card on a P800. |
| ||
| Well for comparasion, as I have a b3d version of the demo running: b3d version, with no textures/lighting:55-65fps average. VividGL-Lite, With textures, no Oct-tree. 110fps average. VividGL-lite with textures, and oct-tree. 150 to 300fps. and that's without lod, which needs rewriting to work with octSurfaces. So, even without lod/oct vivid is nearly 2x as fast a b3d, with it, 5x to 10x faster in some cases. |
| ||
| oh ok, I didn't realise you were talking about Vivid. thought you were talking about the blitz renderer, not GL. Blitz renderer is pretty slow yeah, since you don't have surfae count issues which are farly unique to blitz you shouldn't have much trouble with texture effects :) |
| ||
| thanks for benchmarck (spaced man) for demo vividgl?? Mongia |
| ||
| VividGL? Where can i find info on this, what is it? |
   |