How to store the datas for a large world ?
Community Forums/General Help/How to store the datas for a large world ?
| ||
| Hello, Let's imagine i want to make a game with a large world : Each zone measures 500*500units There are 200*200zones in the world So this makes a world of 100*100km I think this is really a lot of space to fill but i want to generate the terrains procedurally and then add some buildings manually and some buildings procedurally. I see 2 solutions and one of them doesn't seem to be possible because of the huge amount of octets it will take to save the heightmaps and saves : 1st solution : With this solution, the program generates each terrain and saves it as a 512*512pixels heightmap From my understanding, if i save each heightmap with the minimum parameters (PX%,PY%,GreyColor%) in a custom file, this will take 512*512*3*4 = 3145728octets 3.145mo And with this, i have to save the properties of the buildings of the zone, of the objects of the zone, of the characters of the zone, of the nodes and links of the zone. This will take more octets. So let's assume each zone takes 4mo of heightmaps and saves, with a world of 200*200zones this will take 200*200*4 = 160go This is too much ! 2nd solution : With this solution, the program generates the terrain, the buildings, the objects, the characters, the nodes and the links, of the zone only when the player visits this zone. Obviously this will takes less octets and maybe Player will never visit all the zones (200*200 = 40000zones !) But this will require a longer loading time because the program will have to generate the components of the zone. I don't understand how video games like Fallout 3 or Skyrim or GTA4 store their files and saves ? They seem so big and detailed. I only want to make a game with low polys graphics and simple functionalities but it seems to take too much space on the HD. Any suggestions ? Thanks, Last edited 2012 |
| ||
| Ok i have tried to write a custom terrain file by writing only the RGB value as a byte. So now i have a terrain file of 262144octets which is an improvment compared to the 3.145mo ! But still, if the world is 200*200 zones, this will take 200*200* 262144 = 10485760000octets so it is more than 10go ! Do you know another way to store the grey color of each pixel of a heightmap(512*512pixels) ? Thanks, |
| ||
| Multiple options: 1) Just compress the data even more: Each Int is able to hold 4 bytes - which each can be 256 different values. There is no real need to save the PX and the PY-values. If you know where X0,Y0 of the "map-tile" is positioned, you know where X1,Y100 is located... So your Heightmap just stores the "Z-Coordinate". In your case this is a int. If have an absolute minimum in your world (eg. z = -10000) you can add 10000 to the Z-coordinate and save it only using 3 bytes (google for 3byte->int algorithms). This way you would have an additional byte for custom settings (like rotation or type of that tile). So you end up with 512*512*4 = 1024kb (or with 3 bytes = 768kb) EDIT: if like you posted only bytes are used: it is 256kbytes. Now just compress that data with PNG/Targa and it is really compact concerning disk space (memory is still 1024/768/256kb). 2) do it like image compressors take your 512x512 area and count the values. The most often used value is the base for the whole area. Just write the parts differing from the base. If using a base style (snow...) you will end up saving some bytes. -> wont do that as it is not the best thing to do. 3) search for rects. This is the method PNG and others use: they search for patterns in your 512x512 area. Means: wander from x0,y0 to x511,y511 and look for areas bigger than just 1x1. Instead of saving each 1x1-area, they save: x4,y7 = rect(50,20). This is highly optimizing "boring" landscapes... eg wide grasslands. 4) "vectorize" (mathematics) your landscape Instead of saying: grassland height 1, grassland height 2, ... grassland height 10, mountain height 11, ... You safe: gradient(grassland, mountain, height1, transHalfAt 10, height20) That is kind of your "procedurally generation"-method. But instead of saving the result, save the procedure. bye Ron Last edited 2012 |
| ||
| I may be wrong, but I think that games like Skyrim are "Asset drived". Which mean each object you see in the game has been modeled to get the best control and details on it. Then a database (SQL, XML, etc.) is used to store position, rotation & settings of objects. They probably create their own terrain editor, something like Grome Editor from quad software( http://www.quadsoftware.com/ ) who saves each map on a collada format. Depending on the level of details, UV info, structures, etc. You can expect that a map of 10x10 km may take roughly 10 Mb of hard disk space. Then it's up to you to create the best engine to exploit seamlessly all those map with a gradual level of details (LOD), optimizations, etc. Procedural generated maps is a lot of work, especially if you plan to use so many areas on so much space! It's very feasible of course but you take the risk to reproduce those games with an extremely huge world, always the same type of 3D assets and the same boring missions. I'm totally fan of huge procedural generated world with fixed random seed (so you can share your discoveries like in Elite for example), but only if the randomness has been "humanly populated". I'm waiting the next Starbound impatiently ( http://playstarbound.com/ :-). Last edited 2012 |
| ||
| That is kind of your "procedurally generation"-method. But instead of saving the result, save the procedure. Derron>>This is an interesting concept, thanks. But i think i will keep the 262144octets terrain file because it is the only way i can store precisely the height for each unit. I want to be able to modify the terrain in realtime in order to be able to dig in the ground ! Procedural generated maps is a lot of work, especially if you plan to use so many areas on so much space! It's very feasible of course but you take the risk to reproduce those games with an extremely huge world, always the same type of 3D assets and the same boring missions. Armitage 1982>>I don't plan to make a game as detailed as Skyrim or Fallout 3 ! I am realist. But i can certainly make an open world game with low details, an interesting gameplay, and the desire to explore and discover things. I will try with a smaller world to see what i manage to do. |
| ||
| Do you know another way to store the grey color of each pixel of a heightmap(512*512pixels) This is essentially what's been suggested earlier. Rather than storing 512 * 512 * 4 * 8 bits you only need 512 * 512 * 8 bits at the most for a 512x512 Heightmap (Typical Blitz Heightmaps especially only use a single 8-bit byte for Height anyway when loaded from an image (Red channel. Green and Blue and any Alpha channels are ignored) So the first instance, save the heightmaps as 256 colour images if you are loading them with say, LoadTerrain() Otherwise, if you are 'manually' generating the terrain mesh from values read from a file, ensure you only reade a single 8-bit Byte (or even less if you are so inclined and do not need 255 levels of terrain?) |
| ||
| Otherwise, if you are 'manually' generating the terrain mesh from values read from a file, ensure you only reade a single 8-bit Byte (or even less if you are so inclined and do not need 255 levels of terrain?) I think that's what i have done here : Ok i have tried to write a custom terrain file by writing only the RGB value as a byte. So now i have a terrain file of 262144octets 512*512*1byte (the grey color of the pixel from 000 to 255) And then i create the terrain with CreateTerrain() and ModifyTerrain() Last edited 2012 |
| ||
| Remember to use compression methods... that is why i suggested to save them as grayscale targa/png. bye Ron Last edited 2012 |
| ||
| Hehe, funny you mention "digging in the ground" RemiD, cos that's exactly what I like to use in blitzterrain: "destructibility". I set it so that when throwing grenades you get crude craters, which I think looks pretty cool as a feature. |
| ||
| Yes, being able to modify the terrain heights is a cool feature indeed :) |
   |