Loading "areas" for an open world.

Blitz3D Forums/Blitz3D Programming/Loading "areas" for an open world.

psychicbottle(Posted 2012) [#1]
Ok, so im not sure if this should be in general or if it fits here so move it if you feel like it.


I am creating an open-world fantasy RPG, it is not too far yet but the current state is promising. I am near finished with the map-editor and now am working on creating an engine that can run all of the things that is needed in the game mainly events such as "doors, levers, boxes, area triggers Ect. anywho in this process i was working on doors. i have it set up like this, in the map editor you place a "door", each door has a corresponding "pointer" all this pointer does is show where you will be placed if you walk through that door and what angle you will be looking.

i figured i would just free entitys if they did not belong in the "area" and just delete the types. then i got thinking about walking around in the open part of the world (city to city) should i divide the land into seperate "areas" and as you walk it loads in the areas closeish to you and just makes all items "hidden" till you get closer and dissapear as you walk away and then delete if you get even farther away?


RemiD(Posted 2012) [#2]
If each zone has not too many meshes and textures, you can probably load all meshes, textures, animations, sounds and have an occlusion routine to hide/show only the wanted meshes. You want to check how much RAM the components of a zone take when they are loaded, and decide to load all components of a zone at the same time (when loading a zone), or load/delete components depending on the Camera position and orientation.

Concerning the display of others zones, what i would do is create a flipped box or a flipped cylinder around the active zone and have a texture that represents the components of the others zones around the active zone.

Loading the components of a zone progressively can be complicated, i prefer to use doors/portals to choose which zone to load and which zone to delete.

After that, managing the movement of a character (npc) from one zone to another zone and the saving of its position, orientation is a new challenge.
Instead of trying to save all components of a zone (static and moving) in the same file, it may be a better approach to save only the static components of zone in a file and the moving components temporarily in this zone in another file. I am not sure about this, this is just a suggestion.


psychicbottle(Posted 2012) [#3]
yeah, I have devided most "world items" (everything) into a few types. Static, Door, Container, Effect (animated things like flames) i figure i will possibly set up a whole different setup for characters to manage their movements. I was thinking of a temporary file to save "important" NPCs and "Recent" NPCs. important would imply i am working on a quest involving them or they are main characters and Recent would imply that i was in their area recently or still am, so then it will constantly update the position of these NPCs just not visibly show them in the world to avoid lag.

also kinda moving back, is the "area" system an effective way of having different levels\worlds\areas? the types would kinda work like this

pseudocode:

for i.item = each item

if i\area = current_area$
load(i\mesh)
else
freeentity i\mesh
delete i
endif
Next


KronosUK(Posted 2012) [#4]
Interesting topic. Could you ever get a smooth frame rate with Blitz3d if you were progressively loading and freeing a large number of meshes? Even if you staggered the loading.

Discrete zones sounds alot simpler. Presumably the areas between cities doesn't have to be so detailed so could be bigger.

Having said that I never really liked how Oblivion/Skyrim sort of game has loading screens for when you enter every city, house or shop although i can see the reasons why. I prefer the true open world experience from such games as Gothic.


Rroff(Posted 2012) [#5]
Freeing and loading data from disc on each transition will be slow.

I also reccomend using nested type collections for something like this rather than iterating through even possible item/asset each time.


psychicbottle(Posted 2012) [#6]
That's funny, I actually thought of that when i was working on this.

I'm hoping that i will be able to load zones in slowly -interiors and what im thinking im going to do is in cities, the range at which things load and appear will drop greatly but you shouldnt be able to tell since outside the city will be unseeable in most cases due to walls and buildings, as for loading things in on the terrain i think i am going to divide the physical terrain into a few different parts and make three versions of each part. (High LOD, medium LOD, and low LOD) that way i could load in a lot of terrain without concern of lag, i'll probably do the same thing with tree's and boulders.


RemiD(Posted 2012) [#7]

I never really liked how Oblivion/Skyrim sort of game has loading screens for when you enter every city, house or shop


KronosUK>>
Many good games have used loading screens, this is a simpler way to manage the loading/freeing of the components of the zones.



psychicbottle>>You may be interested in this topic :
http://blitzbasic.com/Community/posts.php?topic=97978#1141431


psychicbottle(Posted 2012) [#8]
I have looked into the topic you posted a link to ad found it semi helpful, i still have a few problems. i could not use the "portal node" idea except for cities in which case im game for, id rather have a game that has good gameplay and loading screens for towns than no loading screen and crappy gameplay Prioritize! I am using types instead of arrays so i believe that should work just as well still, but what do you mean by "nested type collections" i have heard the term but am un aware of it's use.


EDIT: how would you suggest i do your "screenshot" idea from what i see is a series of pictures that form a skybox? which i can honestly say i have yet to handle skyboxes.

Last edited 2012


Leon Drake(Posted 2012) [#9]
for static objects you could probably build the meshes in game rather than load them from a file, this way you can thread the loading operation and get seamless open world and game play speed. For objects that take longer to load you can place invisible blocks in the place where it would be so the area is obstructed and you dont get something that will appear and cause you to get stuck.


RemiD(Posted 2012) [#10]

what do you mean by "nested type collections" i have heard the term but am un aware of it's use.


Let's say i have several subzones in my zone, all the components of the zone are already loaded in memory but to keep a good fps i want to show only the active subzone (with its components) and the subzones connected to this subzone (with their components) and update only the moving entities or the entities with sensors in these subzones.

Let's say my zone is divided in 100 subzones.
Let's say there can be up to 100 components that require an update in a subzone
If i don't care about the fps i can do :
ZId% = 1 ;i know the active zone because i have loaded this zone
For SId% = 1 to SubzonesCount(ZId)
 For CId% = 1 to ComponentsCount(ZId,SId)
  ;hide/show component
  ShowEntity(ComponentMesh(ZId,SId,CId))
  ;update component
  ComponentEnergy(ZId,SId,CId) = ComponentEnergy(ZId,SId,CId) + 1 ; add energy
  MoveEntity(ComponentMesh(ZId,SId,CId)) ; move the entity
 Next
Next

This will be 10000 checks...

Now let's say i use the parent child feature :
ZId = 1 ;i know the active zone because i have loaded this zone
SId = 15 ; i know which is the active subzone because i know where is Player
For CId% = 1 to ComponentsCount(ZId,SId)
  ;hide/show component
  ShowEntity(ComponentMesh(ZId,SId,CId))
  ;update component
  ComponentEnergy(ZId,SId,CId) = ComponentEnergy(ZId,SId,CId) + 1 ; add energy
  MoveEntity(ComponentMesh(ZId,SId,CId)) ; move the entity
Next

This will be only 100 checks.

I am not sure if you can achieve the same thing with Types, maybe another coder will be able to show that. I prefer to use arrays except for particles.




how would you suggest i do your "screenshot" idea from what i see is a series of pictures that form a skybox?


Yes that's the idea.

Last edited 2012

Last edited 2012

Last edited 2012


Rroff(Posted 2012) [#11]
There is a way to do type collections which allow you to use nesting type functionality to avoid processing items/entities that you don't need to touch - I'll post up a simple example in a bit - but it can be extended to be quite powerful.


psychicbottle(Posted 2012) [#12]
@ RemiD Honestly i am kinda lost, but couldn't i just add a parameter of "area" to a type and then a level of importance parameter such as "importance"

e.g.

for s.static = each static
if s\area = current are then update
if "distance to area" is > viewdistance#
if "distance to area" is > "where LOD drops" and < ViewDistance#
s\mesh = lowLODmesh
if s\importance = 1 then "keep updating object"
endif

sorry that it's so sloppy but you get the jist

Last edited 2012


Rroff(Posted 2012) [#13]
Heres and example of what I was saying - its a little bit complicated but once you get your head around it its possible to save a lot of processing time by only handling relevant data using zoning, without having to check every instance of an object to see if it should be handled or not:



This example uses tables and chairs to demonstrate - where each table will have a collection of chairs that belong just to it and you can list and do stuff to each of these chairs without ever touching the chairs belonging to another table.

For proper use you'd create function to handle the type stuff thats used over and over again, this is especially useful for UIs implemented in B3D as you can handle the controls within a window without having to iterate through every instance of a control and check if its connected to the window your dealing with (which is a massive performance hog). More specific to this thread you can use a zoning system to group objects within a zone and only ever process the ones relevant to the current zone(s).

Last edited 2012


RemiD(Posted 2012) [#14]
psychicbottle>>Oops i forgot to add something in my previous code example, read it again maybe it will be easier to understand this time.


RemiD(Posted 2012) [#15]

couldn't i just add a parameter of "area" to a type


Of course you can do this, but you will have to loop through all instance of the Type, so if you have 100 subzones with 100 components in each subzone it will be a loop of 10000 instances instead of only 100 in the active subzone (+ a few hundreds more if you want to update the components of the subzones connected to this subzone.)

If you don't care about optimisation, do as you say.

Last edited 2012


RemiD(Posted 2012) [#16]
Rroff>>Ok but with this method you can't access a specific instance with its Id without a loop ? Or can you ?

Type TSubzone
 Field Id%
 Field Component.TComponent
End type

Type TComponent
 Field Id%
 Field Name$
 Field Mesh
 Field State%
 Field Health#
End type


Let's say there are 100 subzones in the zone.
Let's say there are 100 components in each subzone.

Now how do you access directly (without a loop) the Subzone with the Id 76 ?

Last edited 2012


Rroff(Posted 2012) [#17]
Working around that is more of a design issue and probably beyond the scope of a post here. I actually use a much more advanced version of this method of using types where I only have 1 core type and have a database style system with a data tree and data tables and use broadphase filtering to get to what I want without lots of additional checking.

Simplest way if you have a hard coded limit on the number of subzones is to link them from a blitz array - the reason to (mis)use types in the way I posted the example for is that you can have an unlimited number of child instances in a group without having to do any housekeeping of your arrays or chains within B3D but leave it to the C++ runtime side of B3D which is faster.


psychicbottle(Posted 2012) [#18]
would it not save just as much processing to check each instance to see if it is in the area and if it is then keep updating it and if it is not then just ignore it? so i would not be checking 1000 times exactly i would check for the area and wait essentially for a true or false return and if it gets false i just drop it so i dont keep checking it,


Rroff(Posted 2012) [#19]
You'd still potentially be checking 1000s of objects for true/false which on its own is a massive CPU cycle sink.


psychicbottle(Posted 2012) [#20]
hmmm fair enough, i shall look into all this, thanky you both very much


Leon Drake(Posted 2012) [#21]
ya for this kind of issue is why i was looking into using sqlite for my projects.


_PJ_(Posted 2012) [#22]

Now how do you access directly (without a loop) the Subzone with the Id 76 ?


You don't even need the ID field per se, since you might as well just use the Handle() and Object() commands.

i.e:
Local ID=76
Local AccessedZone.TSubZone=Object.TSubZone(ID)
Local AComponent.TComponent=AccessedZone.TSubZone\TComponent

PrintComponentID=AComponent.TComponent\ID



Rroff(Posted 2012) [#23]
^^ Its a design thing tho at the end of the day i.e. for this kind of thing I'd be using some type of scene graph like implementation ( http://en.wikipedia.org/wiki/Scene_graph ) instead of specifying a hard ID like 76 I'd be transversing the data set starting at a fairly limited number of broad coverage nodes and drilling in deeper through a couple of layers until I got down to the set of data I wanted.


RemiD(Posted 2012) [#24]
A possible way to store the properties of the zones and to know which zone to load depending on where Player is going : (This is the approach i use for one fo my project)

Let's say the world of the game has a width of 200 zones and a depth of 200 zones, a zone measure 512*512units (in order to be able to use a terrain)
The zone with the coordinates 0X,0Z (in zones not in units) will have the filename "000X000Z.zone"
The zone with the coordinates 23X,48Z will have the filename "023X048Z.zone"

Then when the Player reaches the border of a zone to go to another zone, it is possible to use the coordinates of the current zone and the orientation and movement of player to decide which zone to load.
For example if player is in the zone 10X,90Z, the zone with the filename "010X090Z.zone" is loaded.
If player orientation and movement shows that player moves toward the zone at front, once the player reaches the border of the zone, the program will save the properties of this zone, and will load the zone with the coordinates 10X,91Z so it will use the filename "010X091Z.zone"

For my game i don't use a "progressive" loading/deletion of the components of the zones because it seems really complicated to do and i don't want to have to think about that for now, so what i do is that i show a message that asks the player if he wants to go to the next zone or to the map screen (in order to select another zone where to travel)

As discussed previously, the other challenge is to manage the update and the storing of moving entities like characters and projectiles who can go from one zone to another zone.
I don't know if this is the best approach, but i always update the characters and the projectiles depending on their distance from player.
If a character is too far from player (one zone or more), i warp it to a node where there is an activity he can do.
If a projectile is too far from player (one zone or more), i delete it.

Another approach is to not use coordinates to delimit zones, but portals, it is easier but many areas will have to not be accessible (blocked by terrains or walls)



psychicbottle>>To understand the "subzone" concept, imagine there are 3 areas in a zone, a village, a farm, a cemetery,
Each subzone has buildings, furnitures, objects, triggers (switches, levers, handles, locks, traps), particles emitters
All furnitures, objects, triggers, particles emitters are childs of a building (the subzone parent)
You know were player is, for example in the village, so you only have to update the components of the village, the near characters, the near projectiles.
The components of the farm and of the cemetery don't need to be updated.

You can also use this concept to choose how to render each area.
If player is in the village, the high detailed buildings will be shown, all the furnitures, objects, triggers, particles emitters will be shown and active.
Whereas the high detailled buildings, the furnitures, the objetcs, the triggers, the particles emitters of the others areas will be hidden, and only a low detailed version of the buildings will be shown.

Last edited 2012


_PJ_(Posted 2012) [#25]
^^ Its a design thing tho at the end of the day i.e. for this kind of thing I'd be using some type of scene graph like implementation


Well I onl.y specified '76' as that was the example given by RemiD and since I have no idea how he is loading his scene-graphs or otherwise obtaining the data, I could only provide a sample of code that would be able to ret=rieve the correct (in this case, specified as #76) by providing '76' as the value for ID and defining it as a Local variable with a literal value.

In "real terms", ID would be populated from the Scene-graph or data file etc. in a manner such as this:

Function LoadLevelData(Level=1)
Local DataFile=OpenFile("Level "+Str(Level)+".dat")
Local DataBytes
While (Not (Eof(DataFile)))
   DataBytes=ReadInt(DataFile)
   ID=DataBytes And 255
   CoordX=(DataBytes Shr 8) And 255
   CoordY=(DataBytes Shr 16) And 255
   Flags=(DataBytes Shr 24) And 255

   BuildLevel(CoordX,CoordY,ID,Flags)

Wend
CloseFile DataFile
End Function LoadLevelData

Function BuildLevelZones(X,Y,ID,Flags=0)
Local AccessedZone.TSubZone=Object.TSubZone(ID)
Local AComponent.TComponent=AccessedZone.TSubZone\TComponent
;...
End Function



Last edited 2012


RemiD(Posted 2012) [#26]

Local ID=76
Local AccessedZone.TSubZone=Object.TSubZone(ID)
Local AComponent.TComponent=AccessedZone.TSubZone\TComponent

PrintComponentID=AComponent.TComponent\ID



_PJ_>>Your example does not work if i create several types and if i create several instances of different types without a particular order.

Graphics3D(800,600,32,2)
SetBuffer(BackBuffer())
SeedRnd(MilliSecs())

Global SubzonesCount% = 0
Type TSubzone
 Field Id% 
 Field Mesh
End Type

Global CharactesCount% = 0
Type TCharacter
 Field Id%
 Field Mesh
End Type

For i% = 1 To 100
 SubzonesCount = SubzonesCount + 1
 S.TSubzone = New TSubzone
  S\Id = SubzonesCount
  S\Mesh = CreateCube()
  NameEntity(S\Mesh,"SUB"+Handle(S))
 CharactersCount = CharactersCount + 1
 C.TCharacter = New TCharacter
  C\Id = CharactersCount
  C\Mesh = CreateCube()
  NameEntity(C\Mesh,"CHA"+Handle(C))
Next

H% = 76 ;75 corresponds to a Subzone, 76 corresponds to a Character, 77 corresponds to a Subzone, etc...
S.TSubzone = Object.TSubzone(H)
 Print("S\Id = "+S\Id)
 Print("S\Mesh = "+S\Mesh)

FlushKeys()
WaitKey()
End()


Here we can see that the Handle 76 corresponds to the instance of a character and not one of a Subzone, so you can't access a Subzone with the Handle 76 because it corresponds to the Handle of a Character.

Or maybe i haven't understood your example. If so please give more details.
Thanks,


Rroff(Posted 2012) [#27]
The integer handle of a type is specific to that instance of the program. You'd have to compare against a field value of the type object for reliability - with the proper design implementation tho you shouldn't ever be iterating through a large number of objects to find the one you want.

i.e. with the type setup I'm using I'd have say 5 large coarse zones - find which the player is inside and then have another 5 smaller zones within that larger zone and again find which the player is in and so on until I'm down specifically to the sub-set of data I want without ever touching any of the content of irrelevant zones within the B3D code ever.

Last edited 2012


_PJ_(Posted 2012) [#28]
_PJ_>>Your example does not work if i create several types and if i create several instances of different types without a particular order.



Well of course my example was purely to show that you COULD just porovide a value (say, 76) and retrieve the object with that reference.

Using ID values which represent different custom object types is certainly gooing to make things a little more difficult, and I'm not sure why you would.,

If the ID field is necesary to identify the object type, then Perhaps you need a different approach altogether - you will never be able to resolve which object is which type out of order if there's no specific means to identify which object types have which ID...