The Occlusion challenge

Blitz3D Forums/Blitz3D Programming/The Occlusion challenge

Rob(Posted 2003) [#1]
The last real frontier in Blitz is still occlusion/vis schemes. Get cracking lads! this is something we all need so suggestions and discussion welcome! :)


Al Mackey(Posted 2003) [#2]
Um.. hold on. Let me go check m-w.com.

Main Entry: oc·clu·sion
Pronunciation: &-'klü-zh&n
Function: noun
Etymology: Latin occludere
Date: circa 1645
1 : the act of occluding : the state of being occluded


Wow. That helped. What do you mean by occlusion?


Ross C(Posted 2003) [#3]
Hiding Entities or triangles that are not in the camera's viewport. Can provide a great speed up and allow more detail on-screen if done correctly.

My attempt was basically to hide any entity that returned false for EntityVisible. Pretty primitive and i'm sure there are alot of drawbacks to it, but it works for what i need it. Won't really know how to go about anything more detailed :)

Another point slightly OT. When a triangle has 3 vertexs with an alpha of 0, i assume it isn't even considered for being drawn on screen? (IE. the same as using hideentity)


Al Mackey(Posted 2003) [#4]
I'm pretty sure Blitz does this automatically to a certain degree. I had an app where a model had all its vertices controlled by child pivots (bones in LightWave terminology). In the first version the child pivots were unparented from the object itself and allowed to be moved independantly.. if they got too far from the object -- that is, if the original bounding box of the object was completely off screen -- the children would all disappear. Once the parent object was moving along with its children, the disappearing bug went away.

This suggests strongly to me that Blitz does its own occlusion based on the origin and bounding box of an object.


Rob(Posted 2003) [#5]
Thats not useful. Blitz CULLS*, it doesn't occlude. This means massive overdraw on levels. For example, you go inside one of 100 buildings, walk into a room and look at the furniture.

Right now, at least half of all that will be drawn. With occlusion, you cut down on overdraw. It really is important.

* culling is a simpler removal based on what isn't immediately within the camera view.


Ross C(Posted 2003) [#6]
Whats the best way to go about doing this? Obviously bounding boxes are a very good way to do it, especially if an entity is a pillar shape and is rotated 45 deg.

Can you actually cull triangles in a surface? I'd imagine there has got to be some sort of balance needed. You don't want a routine that's too processor intensive as the speed you would gain from culling would be taken up by the time it takes to calculate the culling.


Al Mackey(Posted 2003) [#7]
Ah, I see...

Well, one way to do that sort of thing is to have a good typing system in place, where each room has a type entry, and each object has one too that associates it with a room.. then, keep track of which room the player is in, which rooms are visible from what other rooms, and hide any rooms (and their dependant objects) that aren't visible from the player's room.

I'm doing something like that in a game I'm working on, except the doors between rooms are always solid, so only one room has to have its objects visible at once, unless the player is moving from room to room.

I know that's not as nice as smart occlusion, but it's a quick and efficient way of making players believe that the world they're in is large and detailed.


Ross C(Posted 2003) [#8]
Sounds like a good idea, but my world is open and you can usually see most of it at the one time.


Rob(Posted 2003) [#9]
Mark did have a plan with maplet 2: he wanted maplet to be able to cull entities and polygons inside the maplet level: so maplet would actually control whats visible.

However he hasn't had time for this so I am seeking other methods. Halo and I both did a version which coloured entities and checked but it was too slow to precalculate.

Note that all useful occlusion routines will require precalculation to be effective...


Pepsi(Posted 2003) [#10]
I got another portal engine design in the works. It will be made public if it turns out to be at least good enough to use as I am counting on it to be faster than any of my previous attempts. I hope it turns out good because I really want to see something like this to be free for blitz community use.

The only main problem I see with using a "portal only" engine is that it's best to place portals manually in which that doesn't really bother me. I just have to build a "portal editor" for it( And there are differents ways to go about that). It should even work with Quil3D's portal stuff, hopefully, as long as it uses standard like portals(defined as 2d convex/concave shaped mesh in 3D space). Also, It should even work with Maplet if it evers gets the portal update that Mark talked about a while back.

Obviously, every portal engine is different but one thing that usually remains constant between them is the standard defined portal that a portal engine acts upon( ie: as in how it uses that manually placed portal). My current engine design takes the route of if the current room's portal is in view then render it's room ( cell or sections, whatever you want to call it ) on the other side of it. The next step it takes is if any portals in that other room is in the camera's viewing frustum and can be seen through the portal in your current room then the engine will render it's room. Ofcoarse, this is done in a recursive fashion. Actaully, one could simply skip the second step all togethor to make the engine very simple and crazly fast at the cost of level design limitations.

It follows the same principles of the first portal engine that I made in Blitz3D as seen in my link below and also with what I was doing for the Community OpenSource: Project Plasma FPS. The difference is that this time I won't be using any sort of LinePick algorithm. I just need to get my test level done( using gameSpace :P ) so I can hurry up and finish my engine.


IPete2(Posted 2003) [#11]
Rob,

What about the demo of Portals in the Quill package?

Do you think there is a fast way to achieve this through programming in Blitz? Or will it be a combination of a map/level editor and Blitz?

As I understand it the BSP route requires a 'no-license required' editor and complier so unless there's a free unlicensed BSP editor/compiler available to us...

Is there a route through Halos' format CSM and CShop?

The way Virtools does it, I believe, is to have a list for what is to be rendered, this list is culled before the final list for rendering is produced, this is all that is rendered - every frame though!?!

If this was to happen in Blitz, do you think Mark may have to 'attack the problem' from his end for the sake of speed of calculation?

IPete2.


Physt(Posted 2003) [#12]
As far as I can tell, the BSP support in Blitz renders the potentially visible set. Normally, the potentially visiable set used as the first step in determining what is actually visible and thus rendered. So Blitz BSP support is out...

Rob - I assume you are refering to some sort of general purpose occlusion system like that used by Renderware.
http://www.renderware.com/partner_products.htm





RenderWare VisionFX

RenderWare VisionFX is an advanced visibility technology plugin that's essential if you're developing games with large and dynamic worlds. Genres that specifically benefit from VisionFX are Role-playing games, Action-adventure games, First-person combat games, Massively multi-player online games and Urban / city-based games. RenderWare VisionFX works by identifying the visible objects in a world as quickly as possible.

RenderWare VisionFX is available as a plugin or standalone product for PC, Xbox and Nintendo GameCube.

RenderWare VisionFX Features & Benefits

Faster Rendering Times
Once RenderWare VisionFX has done its work, you continue by only drawing the visible objects. This can lead into immense savings in rendering time of up to 400%.

Faster Development Cycle
Development cycles become more efficient as RenderWare VisionFX calculates visibility in reducing the time required for static visibility calculation, using PVS for example. The artists creative restrictions are reduced and export processes can also become simpler and faster.

Increased Scene Richness
RenderWare VisionFX enables more characters and objects, with greater richness in scenes. It removes visibility-based creativity constraints so that you can make your world truly alive with interaction (e.g. deformable environments). New levels of realism can therefore be realistically achieved and larger worlds are handled with much less effort.

Increased Freedom in MMO Game Worlds
As RenderWare VisionFX is a real-time visibility solution, developers concerns over initial world size are greatly reduced. Developers can also allow players to introduce their own scenery (for example players’ homes and businesses) into the persistent world. And because RenderWare VisionFX reduces the pre-processing required for level design, expansion packs can be introduced easier and with a greater focus on the content.

Cutting Edge and Feature Rich Technology
RenderWare VisionFX effectively blends a number of cutting edge visibility technologies including hierarchical view culling frustum, occlusion culling, portal culling and contribution culling. RenderWare VisionFX also supports on-demand loading of scene data using the object visibility information, enabling the application to effectively stream in visibility optimized content. Other techniques such as LODs, multi-resolution geometry and image-based rendering can be used with RenderWare VisionFX for further enhancing performance and content scalability.

About RenderWare VisionFX Technology
RenderWare VisionFX is the first efficient general-purpose visibility optimizer that requires no scene pre-processing and works with massive dynamic environments of any topological structure. The visibility queries are output-sensitive, so the amount of time RenderWare VisionFX spends in finding the solution is dependent on the number of visible objects rather than the number of objects in the scene. RenderWare VisionFX calculates visibility in reducing the time required for static visibility calculation.
RenderWare VisionFX is powered by Umbra™ from Hybrid Graphics - The leaders in advanced solutions for real-time computer graphics.


Powered by Hybrid Graphics’ Umbra™ technology, VisionFX is available for PC, Xbox™ and NINTENDO GAMECUBE™, as a plugin for RenderWare Graphics or as a standalone library.






Ross C(Posted 2003) [#13]
Here's a wee VERY simple demo for some vis stuff.

Press keys 1 and 2 to switch between the over head cam and the normal one. Arrow keys to move.

Doesn't take into account objects that are behind other objects.

Graphics3D 800,600
SetBuffer BackBuffer()


camera=CreateCamera()

camera_overhead=CreateCamera()

PositionEntity camera_overhead,0,60,0
HideEntity camera_overhead

sphere=CreateCone()
ScaleEntity sphere,1,1,3
EntityParent camera,sphere
PositionEntity camera,0,1,0


PointEntity camera_overhead,sphere

light=CreateLight()

Type cube
	Field ent
	Field x#,y#,z#
	Field hidden
End Type


For loop=0 To 20
	c.cube=New cube
	c\ent=CreateCube()
	c\x=Rnd(-20,20)
	c\y=0
	c\z=Rnd(-20,20)
	PositionEntity c\ent,c\x,c\y,c\z
	c\hidden=0
Next


While Not KeyHit(1)



	If KeyDown(200) Then MoveEntity sphere,0,0,0.2
	If KeyDown(208) Then MoveEntity sphere,0,0,-0.2
	If KeyDown(203) Then TurnEntity sphere,0,1,0
	If KeyDown(205) Then TurnEntity sphere,0,-1,0
	
	If KeyHit(2) Then
		ShowEntity camera
		HideEntity camera_overhead
	End If
	If KeyHit(3) Then
		ShowEntity camera_overhead
		HideEntity camera
	End If
	
	Gosub checkvis
	UpdateWorld
	RenderWorld
	Flip
Wend
End

.checkvis
	For c.cube=Each cube
		If EntityInView(c\ent,camera) Then
			If c\hidden=1 Then
				ShowEntity c\ent
				c\hidden=0
			End If
		Else
			If c\hidden=0 Then
				HideEntity c\ent
				c\hidden=1
			End If
		End If
	Next
Return



Ross C(Posted 2003) [#14]
Does EntityAlpha 0 have the same effect as hideentity? I mean except for collision dection, are they just the same?


Pepsi(Posted 2003) [#15]
Yes, the entity's polys are not passed to the render with hideentity and EntityAlpha 0.


Ross C(Posted 2003) [#16]
Good stuff!


Koriolis(Posted 2003) [#17]
@Ross C: you know that blitz already does this, right? Rob is talking about occlusion, what you showed is only a clipping against the viewing frustum.


IPete2(Posted 2003) [#18]
Ross,

Easy when you know how eh?

btw I tried to globalise the cameras and put it into a function, works fine there too.

IPete2.


Pepsi(Posted 2003) [#19]
Ross has a good step in a occlusion process. It's basically how I do it with the "rooms" of my level. Lets take these cubes for an example. Imagine those cubes connected to each other and the part on the sides that cubes touch each other are eleminated where you can run around inside them. Well, we want the cubes that are in the viewing frustum but are totatlly hidden behind other cubes not to be rendered. Thus, I would use EntityAlpha 0 on those specific cubes to keep collision data so any entities in them effected by gravity/moving around wont wonder into the void.

The EntityInView used in this case isn't just used to do what Blitz already does, but to make sure the rooms(cells, sections, or whatever) that come into camera view but not visible to your eyes to not to be rendered. That's what I see in Ross's program above.


Ross C(Posted 2003) [#20]
I see what your saying Koriolis :) This is a bit of learning for me here.

@Todd

Thanks :o) Well, i think that would be the first step if i were to do occlusion, get rid of all the stuff that wasn't needed. I don't really know where to go from there. Maybe entity in view would be better, again tho it wouldn't be a very fullproof idea, as i think that everything has to be pickable and it doesn't work too well:

Graphics3D 800,600
SetBuffer BackBuffer()


camera=CreateCamera()

camera_overhead=CreateCamera()

PositionEntity camera_overhead,0,60,0
HideEntity camera_overhead

sphere=CreateCone()
PositionEntity sphere,0,-1,0
ScaleEntity sphere,1,1,3
EntityParent camera,sphere
PositionEntity camera,0,1,0


PointEntity camera_overhead,sphere

light=CreateLight()

Type cube
	Field ent
	Field x#,y#,z#
	Field hidden
End Type


For loop=0 To 20
	c.cube=New cube
	c\ent=CreateCube()
	c\x=Rnd(-20,20)
	c\y=0
	c\z=Rnd(-20,20)
	PositionEntity c\ent,c\x,c\y,c\z
	c\hidden=0
	EntityPickMode c\ent,2
Next


While Not KeyHit(1)



	If KeyDown(200) Then MoveEntity sphere,0,0,0.2
	If KeyDown(208) Then MoveEntity sphere,0,0,-0.2
	If KeyDown(203) Then TurnEntity sphere,0,1,0
	If KeyDown(205) Then TurnEntity sphere,0,-1,0
	
	If KeyHit(2) Then
		ShowEntity camera
		HideEntity camera_overhead
	End If
	If KeyHit(3) Then
		ShowEntity camera_overhead
		HideEntity camera
	End If
	
	Gosub checkvis
	UpdateWorld
	RenderWorld
	Flip
Wend
End

.checkvis
	For c.cube=Each cube
		If EntityInView(c\ent,camera) Then
			If EntityVisible(camera,c\ent) Then
				If c\hidden=1 Then
					ShowEntity c\ent
					c\hidden=0
				End If
			Else
				If c\hidden=0 Then
					HideEntity c\ent
					c\hidden=1
				End If
			End If
		Else
			If c\hidden=0 Then
				HideEntity c\ent
				c\hidden=1
			End If
		End If
	Next
Return


Entities disappear even tho they can still be seen.


Ross C(Posted 2003) [#21]
What about using a grid system? Placing each level item in a grid and using some 2d maths to figure out if it should be viewed or not.

...t.......
.######....
.#.........
.#..p......
...........


Find out if p can see t. You would need unhide the entity one grid square before they are actually seen tho, so it doesn't pop into view.


Pepsi(Posted 2003) [#22]
Yeah, I wouldn't use entity visible since it only operates on the center point of an entity. That's where portals come in handy here. Using cubes as an example, I perfect example would be how that game Descent ( 1 & 2 ) used portals. Imagine two cubes next to each other where the sides that the two cubes touch share the same vertices. Instead of having polygon sides there, use that vertice data for portal data. You can imagine those sides as a portal where it's now invisible. Say we are standing in one cube and the side that is now a portal is in our camera's viewing frustum. So with preconfigured portal data, the engine would take the portal data from that portal we see in view and display the cube that is next to the cube we are standing in.

Edit: didn't see above post. But, what you said above with your grid stuff can be done with a "trigger box". You can select all rooms that can be seen from one room manually or find some complex pre-calculation to do it for you. Then have a "box" or something in the doorways of each cube ( or room ) and set states to AlphaEntity 0 or 1 other rooms when you walk/run through them. But that wouldn't be precise as portals are. And ofcoarse you can go the BSP route as stated below.


JoshK(Posted 2003) [#23]
The Singularity engine now splits the world up into single faces, each with a mesh and a plane equation. Compiling the BSP isn't that hard, you just check which side of the plane a wall is on, then go to that wall and repeat with another unassigned wall. There are a couple of things I have to do before it's really functional.


Ross C(Posted 2003) [#24]
I think this stuff maybe a wee bit too advanced for me right now. But i'll give some of the suggestions a try. Thanks for your help ppl!


Pepsi(Posted 2003) [#25]
np ... ofcoarse this is not just about portal's and bsp's, if you find another solution, the community would be grateful if you decided to share the knowledge. Good luck! :)


Ross C(Posted 2003) [#26]
cheers :)


Koriolis(Posted 2003) [#27]
@RockStar: wouldn't having each face as a single mesh compltetly defeat the purpose? Todays card are a lot more comforable drawing 1,000,000 polygons in a shot than 1000 series of 1000 polygons, not to say 1,000,000 serie of one single polygon. Gfx card really don't like frequent state changes.
That's why blitz doesn't handle a lot of surfaces very well, and is also the reason why in Quake 3 (or is it Doom3?) the BSPs are not used any more for rendering, but only for collisions purpose.

I *think* for todays cards the best is to use some lazy occlusion system with octrees or something similar. BSPs are just not very well suited for high poly count levels (they generate a lot of additional faces due to splitting, and in addition the PVS size increases very fast with the poly count increase).
Any thougts on this?


John Blackledge(Posted 2003) [#28]
"I wouldn't use entity visible since it only operates on the center point of an entity" - ?
Pardon?
I have implemented the previously suggested 'If Not EntityVisible then HideEntity' type code.
I have huts which as I turn appear at the side of the screen, as you would expect, but the centre point of the entity is still offscreen....


Shambler(Posted 2003) [#29]
Strange, the docs say that with a mesh EntityInView checks the bounding box of the entity for visibility.

I don't see this happening, it only appears to use a point, maybe we have to set up EntityBox but the example in the online docs doesn't.

I'll have to try this at home since I'm at work atm.


Pepsi(Posted 2003) [#30]
EntityVisible ( src_entity,dest_entity )

When the last time I played around with it, it uses the center point of an entity to see if they can "see" each other or not if another entity is obscuring the view between them. In this case, I'm referring to being inside cubes( image their faces inverted inwards as if you are walking around inside of an indoor level) that are connected to each other as where the sides that touch each other are gone. If I use EntityVisible to see if another cube is visible, you can easliy be in a positon where you can see another part of a paticular cube, but the center of it could be behind another cube. That second piece of code RossC shared with us showed by using the EntityVisible command that his cubes where showing and dissappearing only when the center of those cubes where being obscured from another cube.

Sorry if I didn't make myself clear.


Rob(Posted 2003) [#31]
EntityVisible uses the centerpivot of a mesh. EntityInView uses the bounding box.


bradford6(Posted 2003) [#32]
the point is...

an octree/bsp style occlusion scheme is necessary for most 3d game levels.


col(Posted 2003) [#33]
Yep. I agree, occlusion will also allow some massive polygon game worlds to be present but because of the occlusion you will only need to render a small part of that game world, hence a small number of polys ( or entities ).

A technique I'm working on at the moment which is suitable for indoor game worlds is doing all the occlusion in a pre-processed fashion using manually placed portals. You can store all the vis data in a list ( adjacency list ). Each element of the list could be a seperate entity ( room for eg. ) that stores adjacency info ( that is other entities that are visible from this one ). Then, in game, look at what entity the player is in and refer to the adjacency list to render the other potential visible rooms. Blitz will cull entities out of camera shot for you. It would be quicker to access a memory data than doing all those EntityInView/EntityVisible ( internal matrix bashing ) checks.

Food for thought :)


Physt(Posted 2003) [#34]
Isn't occlusion somewhat seperated from culling using octrees and bsp's?

One could use an occlusion step afer getting the potentially visiable set from the octree or bsp. then It would figure out what needs to be drawn and what is behind something else and therefore doesn't need to be drawn.

A good general purpose occlusion system would allow you throw abritray geometry at it and it would only draw what was visible. I know it sounds like the holy grail but such systems exist.


Shambler(Posted 2003) [#35]
Yes, occlusion is done AFTER octree/bsp culling.

You do your culling then you end up with a list of entities which are visible/partially visible or occluded by other entities in the scene.

In Blitz we don't really go down to the triangle level when rendering so I think what we are after is an occlusion technique which is just for whole entitities/meshes.

I've yet to actually need an occlusion system like this though, since graphics cards are so fast these days the octree/bsp/portal culling is enough to get the polycount down to a reasonable level.

I don't think most of us actually need it...that doesn't make it any less interesting to talk about though =)


Physt(Posted 2003) [#36]
http://www.gamasutra.com/features/19991109/moller_haines_01.htm


Rob(Posted 2003) [#37]
You have got to be kidding shambler. You're basically saying that I can't be allowed to put 10x the detail into my levels so it runs well on anything like my sys specs!


col(Posted 2003) [#38]
So for us to have occlusion at the triangle level it really does need to be built into the Blitz rendering pipeline. Correct me if I'm wrong, but it looks like blitz just throws the entities ( not Terrains or BSPs ) into the render engine which only culls away complete entities outside the view frustum, I'm sure that Blitz leaves Direct3D to hide the back facing polys ( hence you dont get a true TrisRendered() ).

Another thing to mention is that say for example we go for an 'entity culled occlusion' which most are getting to work, we mustn't forget that the game still goes on in 'culled sections' of the game so if we are using HideEntity then that entity is no longer involved in the collision engine - EntityAlpha will still be involved in collisions.


IPete2(Posted 2003) [#39]
So how many vertices is 117,800+ triangles?

Because that's how many I've got rendering currently with a camera range of about 15,500.

I know occlusion would help us keep the frame rate up (it's about 20 fps at the moment on an Athlon 2200, with a 64 MB nVidia GeForce 4), but I think Blitz does pretty well to handle this size of a level!

IPete2.


col(Posted 2003) [#40]
No one is knocking blitz at its rendering speed. I'm sure most would agree that Blitz does do a great job. But with some sort of occlusion, 60fps wouldn't be impossible - instead of your 20fps. Great visuals moving smooth and great visuals moving jerky can/will make or break a title.


Rob(Posted 2003) [#41]
I'm pretty sure my hardware wouldn't even be able to play ipete's game :)


Shambler(Posted 2003) [#42]
One major performance bottleneck with Blitz is that it uses the old DirectX Retained mode.

This means where I could be getting 40 million triangles per second with certain geometry, I only get 5-10 million, thats a huge performance hit.

I used the same mode when I started programming with C++ but soon found that to get really fast rendering you had to use immediate mode, alot harder to program but giving you more control and speed at the same time.

Other than the occlusion commands currently available in Blitz I don't see us gaining much more performance with more complex methods of occlusion.

The overhead of going down to triangle level and calculating occlusion will be greater than just throwing the mesh at the graphics card to render.

On a similar note,one thing you could do to gain some performance is render entities from closest to farthest away so you get a performance gain from the Z buffer by reducing overdraw...not sure if Blitz already does that, would need a test =)


Pepsi(Posted 2003) [#43]
Just a side note: Mark Sibly himself corrected me when I said something about blitz3d was using DX7's Retained Mode. He said he is using immediate mode with his own retain mode design.


Rob(Posted 2003) [#44]
Funny how people assume :)


Pepsi(Posted 2003) [#45]
Yeah, assuming always comes back and bites me in the arse.


Not Available(Posted 2003) [#46]
Sorry for the cross post - but this is more relevant here:

I remember doing some stuff with this a while ago, no for culling, but to see whether I had to animate/procedural texture an object only when it was in view - I checked first EntityVisible for the object and then EntityInview for the object if they were both true then we were in business otherwise it didn't matter. How I got decent performance was to run the visibility check every 66 millisecs (why that magic number I can't remember... not documented in the source)...

But onto triangle level stuff...

The Ray_Intersect_Triangle, and Ray_Intersect_Mesh functions in the code archives could be useful for doing triangle level occulsion (coded in C++ in a DLL would be the way to go), again - I'd limit just how much removal the occulsion it could do in it's time (ie. a limit of say 30 ms per occulsion run) in addition to making it force the occusional full run. I'd also make a copy of the original object before I started removing triangles and add some velocity to camera rotates & movement so I could predict whether an object was going to be coming back into full view and use the original copy. Also, depending on how complex the scene was - I'd probably think about storing copies of the occluded mesh at various stages so that they could be re-used without re-occulsion if that makes sense.

eg. just off the top of my head:

Complex mesh in middle of screen
do
cursor left adds velocity to camera rotation
predict worst case scenario (ie. if you took your finger off the key now, what is the x,y,z,rx,ry,rz of the camera - project a box from the camera out to the viewdistance (this is used for a boolean)

trace ray from center camera to four corners of object (or more)
if none are visible then hide object

on next removal call (ie. every N millisecs)
remove triangles based on worst case scenario
via a boolean subtraction of the object - inverse of the box
(the inverse box is like a big cube with a hole in the middle that the camera looks through, everything outside of the hole is subtracted at a triangle level)

loop

I would use a similar method to cull objects partially obscured by other objects - by doing an extrude from the backfaces of an object in a direction paralell to the cameras vector, extrude object to viewdistance, subtract it from any objects it intersects.

We all know about how to do this stuff with trees etc, however I'd be interested to code the boolean subtraction stuff and see how it performs... again it would need to be in a .DLL for speed I think, but a prototype in B3d would be the way to go initially.

Just some ideas; I don't really have a need for this sort of thing at the moment; however I will in the next 6 months - if I crack the code I'll share...

-R

p.s. be forgiving - i'm just banging this out quickly to seed some ideas - and i'm exhausted ;)


Ross C(Posted 2003) [#47]
Ok, moving back a bit in the post. I think blitz is doing something with those entities that aren't on screen. Hiding them is pretty important i think, cause i get a fair speed up if there hidden. This code only hides entities that aren't in the viewport.

Again, key 1 for the normal camera, key 2 for the overhead, arrow keys to move. This time, key 3 for none hiding mode and key 4 for hiding mode.

If you move to the end of the cubes, then turn and don't face them, try each mode. You'll see you get a fair increase. Try about 400 - 2000 cubes.

num_cubes=Input("How many cubes?")

Graphics3D 800,600
SetBuffer BackBuffer()



camera=CreateCamera()

camera_overhead=CreateCamera()

PositionEntity camera_overhead,0,60,0
HideEntity camera_overhead

sphere=CreateCone()
PositionEntity sphere,0,-1,0
ScaleEntity sphere,1,1,3
EntityParent camera,sphere
PositionEntity camera,0,1,0


PointEntity camera_overhead,sphere

light=CreateLight()

Type cube
	Field ent
	Field x#,y#,z#
	Field hidden
End Type


For loop=0 To num_cubes
	c.cube=New cube
	c\ent=CreateCube()
	c\x=Rnd(-80,80)
	c\y=0
	c\z=Rnd(-80,80)
	PositionEntity c\ent,c\x,c\y,c\z
	c\hidden=0
	EntityPickMode c\ent,2
	EntityColor c\ent,100,100,255
Next

mode=0;0 for none vis,  1 for vis

While Not KeyHit(1)



	If KeyDown(200) Then MoveEntity sphere,0,0,0.2
	If KeyDown(208) Then MoveEntity sphere,0,0,-0.2
	If KeyDown(203) Then TurnEntity sphere,0,1,0
	If KeyDown(205) Then TurnEntity sphere,0,-1,0
	
	If KeyHit(2) Then
		ShowEntity camera
		HideEntity camera_overhead
	End If
	
	If KeyHit(3) Then
		ShowEntity camera_overhead
		HideEntity camera
	End If
	
	If MilliSecs()<timer+1000 Then
		frame=frame+1
	Else
		fps=frame
		frame=0
		timer=MilliSecs()
	End If
	
	If KeyHit(4) Then mode=0:Gosub showall
	If KeyHit(5) Then mode=1
	
	If mode=1 Then
		Gosub checkvis
	End If
	UpdateWorld
	RenderWorld
	Text 0,0,"fps="+fps+" mode="+mode
	Text 0,10,"Key(3) mode 0= no hiding of entities that aren't in the cameras viewport
	Text 0,20,"Key(4) mode 1= hide all entities that aren't in the camera viewport
        Text 0,30,"Polys="+TrisRendered()
	Flip 0
Wend
End

.checkvis
	For c.cube=Each cube
		If EntityInView(c\ent,camera) Then
				If c\hidden=1 Then
					ShowEntity c\ent
					c\hidden=0
				End If
		Else
			If c\hidden=0 Then
				HideEntity c\ent
				c\hidden=1
			End If
		End If
	Next
Return

.showall
	For c.cube=Each cube
		ShowEntity c\ent
		c\hidden=0
	Next
Return



Physt(Posted 2003) [#48]
http://www.realtimerendering.com/

Manual has a large section on dynamic occlusion culling
http://www.hybrid.fi/dpvs_download.html


Pepsi(Posted 2003) [#49]
Ross, I agree 100% with you without even having to look at your new code there. I used that fact with my previous portal systems. I don't know why some people don't get that. Hopefully with your code above, they will now.

Edit: Yup, thats a good example.


Mustang(Posted 2003) [#50]
I think blitz is doing something with those entities that aren't on screen. Hiding them is pretty important i think, cause i get a fair speed up if there hidden.


Hiding entities is important at least if you have collisions set up:

http://www.blitzbasic.co.nz/b3ddocs/command.php?name=HideEntity&ref=3d_cat

...So even just for this purpose you should hide stuff that's not on the screen preferably?

For my own game I will try room node occlusion, ie have one "root null" which will act as a base for EVERY static object in the room (wall parts, doors, chairs, pipes, boxes, dead monsters etc) and then construct somekind of list of node visibility (node-to-node).

Hiding the "root null" will automatically hide every child object too, and I can also add objects to that group (null) easily runtime. Hardest part is to make the visibilty list, so far I have no idea how to make it automatic. I'm NOT aiming polygon perfect occlusion, is useless... object based occlusion is enough for me.


marksibly(Posted 2003) [#51]
Hi,

Occlusion is *hard*!

Some observations from experiments I've done:

(1) Hardware evolution shifts the goalposts. I have seen many occlusion demos (including, I think, some from the system renderware uses) where the demos were faster with occlusion turned off! I imagine that when they were written they were pretty quick, but hardware T&L has changed the situation quite a bit. Hardware likes to be given a decent chunk of polys at once, which tends to conflict with...

(2) Occlusion must be done accurately or not at all - close enough is not good enough. As soon as you get a bit too 'generous' with an occlusion algorithm, the effect ripples and you get an exponential slowdown.

(3) There is no one solution. Most modern occlusion algorithms work based on the assumption the geometry is 'highly occluded indoor scenery', but take things outdoors and such algorithms die. Doesn't matter to Carmack as he obviously doesn't go outdoors much, but for a generic game creation package it sucks a bit. Its probably a necessary evil though...

(4) Portals are tricky. The idea is neat - create a temporary 'frustum volume' using the viewpoint and a portal poly and clip further portals to it recursively - but the complexity is exponential. For example, if an average sector has just 3 portals, then each time you traverse another sector you are cubing the amount of work required.It doesn't take many sectors for this to blow up. It then becomes tempting to increase the size of sectors, but again, you start running into problems with (2) above!

(5) Should we even be bothering? While I was writing Maplet, I was unable to come up with a level that slowed down my hunble GeForce2. Hardware is hugely faster now and will continue to improve. Sure, there will always be a scene complex enough where occlusion is a 'win', but how much effort will be required to generate such a scene?

So...perhaps we should be thinking LOD instead of occlusion?


Pepsi(Posted 2003) [#52]
LOD in indoor scenes? Ok, what about if you are at one end of a large indoor level and an AI Bot is at the other end of the large level. If you try to use LOD on the polys furthest away from you in an indoor level, I can only imagine the nightmares on trying to keep animated moving entities effected by collisions and gravity inside the level.

Portals, yes it adds up when there are more portals in each room to check on. But, it's how quick the method of working with portals which is key here. Portals used with sections( rooms, cells, whateva... ) of a level doesn't need to be 100% accurate. If I can get over my programmer's block thing I'm going through at the moment, I'll have a demo with source to show what I mean out here, hopefully, soon.


marksibly(Posted 2003) [#53]
Hi,

I'm talking purely rendering here...physics is another problem altogether!

As for portals, the method had better be damned quick, because you are likely to be processing hundreds of portals in even a moderate scene. No matter how you look at it, the complexity of 'the classic' portal algorithm is great, which is not to say there aren't more efficient variations...


JoshK(Posted 2003) [#54]
Mark, no offense, but the Maplet levels I have seen aren't exactly UT2003. Some of us ARE pushing that many polys. I have maps that will make my GEForce4 crawl without occlusion. I think this is one instance when it would be good to commit to one method and lay down some rules, instead of trying to make Blitz3D completely nonspecific.

Incidentally, there isn't even an efficient way of implmenting BSP into the rendering, since the speed gained by the BSP occlusion would be fighting the speed lost by having every face a separate surface.


marksibly(Posted 2003) [#55]
Hi,

No problems, Mr Rockstar!

But...where are these levels? Can you post a .X model?

[edit]
When I was writing the original occlusion engine, I put a call out for level models to test it with.

There were 2 problems with the stuff people sent me:

1) They had terrible geometry - flipped polys, coplanar polys etc etc. This made them hard to process.

2) None of them slowed down my card! I dutifully spent time trying to algorithmically clean up the models, and learned a lot in the process, but I was always nagged by the 'why am I doing this, they run OK as-is?' thing!

Point 1 lead me to write Maplet - which ONLY generates 'correct' geometry. Point 2 lead me to make Maplet as simple as it is.

This is not a 'Blitz is so great we don't need occlusion' rant - its more a 'brute force is getting pretty brutal' thing.
[/edit]


LT(Posted 2003) [#56]
I agree with Halo - the expectations for level complexity are rising indeed. I have personally worked on levels in the one million poly range and higher!!

If portals are only placed at connections between rooms, the number of them isn't that high. I am referring to a system that requires hand placement, which some may feel is an imposition on the artist. I would argue, however, that it is insignificant compared to building Lods for everything - inside and out!


Pepsi(Posted 2003) [#57]
I agree with these level complexity and portal placement statements. With portal placement, it's needs to be a manual placement job over an automated one... then you wont even come close to processing hundreds of portals at one time. IMHO, manual portal placement will always lead to the best optimized situations. As LT basically said, it's an extra step in building levels but not a big one. Someone else besides that actaul level creator can actaully get in with a custom editor or an option of an editor to place portals on an already built level if needed be.


marksibly(Posted 2003) [#58]

With portal placement, it's needs to be a manual placement job over an automated one



I disagree. Manual placement will break my rule (2) above.

Yet I agree - I think the answer to a descent occlusion algorithm does lie in manual placement of portals, as this allows rule (1) above to work.

Portals may well be the answer, but the current portal algorithm is not 'right'. I've tried it.


jfk EO-11110(Posted 2003) [#59]
My thoughts

Incidentally, there isn't even an efficient way of implmenting BSP into the rendering, since the speed gained by the BSP occlusion would be fighting the speed lost by having every face a separate surface.


Absolutely true when you have a single Mesh Level as a Maplet Mesh.

You might remember my afforts to use some kind of occlusion System. First I worked with a House Model that included tons of Child Meshes. I used a precalculated Visibility List for each Cluster (about some "Meters") to hide or show the Children. This gave me about 40% more Speed, but even then some people with faster Cards said the Occlusion System didn't make it faster on their Machines.

However, here is a Realtime Calculation System that is based on my Identification de Colour Unique (ICU) or unique color indentification system. It is an Adaption by Shadowturtle, Author of the Scream3D Editor. I have no idea how fast it is, but I found it pretty clever:

First all Object must be in a List. Every Loop they are colored using a unique color. It's possible to use right the List index as a Color, but this works only in 32 Bit Mode. Probably it works in 16Bit too when $F-Steps are used.
All Objects EntityFX are set to fullbright and Color instead of Texure Rendering.

Now the Cameraviewport is set to something very tiny, maybe 160*120 or even less.

A Renderworld is performed. Now all Objects in the List will be compared with the Pixels of the Rendered Picture. If their Color is visible, they will be shown, if not then they will be hidden.

After hiding everything neccessary the cameraviewport is set to original Size, EntityFX is reset to Texturing and Lightning and whatever.

Now it continues as usual, a fullsize Renderworld, Flip etc.

As I said, I don't know how efficient this Method is, but I think for Levels with lots of seperate Objects, combined with a relative reasonably short CameraRange it could boost the Speed quiet a bit.

EDIT: Tested - too slow. Precalculation seems to be the only way - combined with halos Idea to split up a Mesh in Surfaces.


But with all those "Render only what's visible" Solutions you have to UpdateWorld _before_ the Occlusion, because when you hide things behind the camera before the UpdateWorld and then walk back, you'll fall through the ground.


Pepsi(Posted 2003) [#60]
How accurate do you want portals? Where they are clipped by polygons in front of them to determine if other portals behind them can be seen through them?

Aight, let me see if I can break me old PSP out and draw some illistrations.


LT(Posted 2003) [#61]
I don't understand what you mean in rule #2, or how this would conflict with manual placement. A portal placed at every doorway is very accurate, no?

Btw, I've tried it also...seemed to work pretty well.


marksibly(Posted 2003) [#62]

I don't understand what you mean in rule #2



The problem is, if you allow people to create, say, 200 poly sectors (which may be optimal for the hardware) and use the 'classic' portal algorithm, none of those 200 polygons will obscure portals. If you can think of a way they *can*, I think you've cracked it.

If you reduce the number of polys per sector, you'll get both exponential blowout of the portal algorithm, and hardware thrashing.


Btw, I've tried it also...seemed to work pretty well.



Again, show me the demo, give me the data...

My P3666/GeForce256 could chew through 20,000 poly levels at 85FPS. God knows what my current setup could do!


fredborg(Posted 2003) [#63]
A friend of mine wrote this thesis on the use of portals in games. There is also some stuff about auto generating the portals, but I havent read it :) Maybe it can be of help somehow!


LT(Posted 2003) [#64]
Ah, ok. That's completely different from the system I am referring to.

What I'm talking about is a scheme similar to what Mr. Riggins had suggested in previous posts. If a level interior can be broken down into spaces that represent rooms, and they are connected with portals, then the visibility of the portals determines the visibility of the rooms.

A "room" would be more like 5,000 polys. I'm not at all convinced that it could not be done algorithmically, but hand-placed is a hell of a lot easier and allows the level designer the flexibility to make those decisions about what can and cannot be seen.


Pepsi(Posted 2003) [#65]
Exactly, LT. Btw, does setting an Entity with AlphaEntity 0 or 1 all the time thrash the hardware?

Thanks for sharing the nice looking thesis, Fredborg. :) I skimmed through it, but didn't see anything about using portals with concave cells. Mat have missed it. Just like to point out that with Blitz3D, it doesn't matter if the cell (section, room, whatever...) is concaved or convexed if a portal algorithm is used like what me and LT are talking about.


Rob(Posted 2003) [#66]
The problem with an algo to determine visibility data is that it has to account for "bad" geometry! open spaces, non-welded gaps and double faces are just part of the problem.

With bsp based vis'ing, you don't have this problem as the levels are watertight. This is why something like maplet is such a good idea. Maplet designed areas would be overlaid by the geometry of any old imported level, yet clip what lied within...


marksibly(Posted 2003) [#67]

it doesn't matter if the cell (section, room, whatever...) is concaved or convexed



It *totally* matters! See rule (2) above...with a convex space nothing can obscure a portal, by definition.

Have you got an occlusion demo I can check out?


Pepsi(Posted 2003) [#68]
"It *totally* matters!"

Nope! :P hehe

I think you are thinking BSP convex stuff.

I'm working on my occlusion demo at this very moment.
-----------------------------------------------------

edit: image deleted
- Red Letters = Room ( yellow lines )
- Green Letters = Portal (Blue Lines)
- Purple Numbers = Camera Positions

In room A at position 1 where the camera is facing straight up towards room D, a simplified portal algorithm can just render/show rooms A,B and D. From position 1, you can only see portal A and you can see portal C through portal A.

If you went down to polygon level where polys could clip portals to help in more detail occlusion, then at position 2 we would not be able to see portal B through portal A because the room B's walls would be in the way.

But, at position 2 with a simplified portal algoritm, portal B can be seen through portal A when the room's wall are not taken into consideration. So room C would get rendered/shown because portal B can be seen through portal A in the viewing frustum. That's a side effect for a simplified portal algorithm, but an ok one to deal with providing we know that for careful manual portal placements.

My demo will only cater for portals that can be seen from your cuurent room that you are in and the portals that you can see through portals with your eye. I'm working on it! :)


Rob(Posted 2003) [#69]
Surely this means you need to set everything up your way, making it useless for the majority of users?


LT(Posted 2003) [#70]
The problem with an algo to determine visibility data is that it has to account for "bad" geometry!

If the portals are placed by hand, then bad geometry is not an issue.

...with a convex space nothing can obscure a portal, by definition

The idea is not to obscure portals, but rather to test portal visibility THROUGH other portals in recursive fashion. e.g. If camera is inside room A and can see portal AB, then test room B's portals to see if they can be seen THROUGH portal AB, render those that can, etc.

Portal occlusion is another possibility, with some additional steps, but what we're talking about is a gross method that is a bit more granular than octree world division.

[EDIT]
Rob, not necessarily. It should be possible to create an editor that would allow for just about any kind of interior level. This is certainly not for the outdoors though!!
[/EDIT]


Pepsi(Posted 2003) [#71]
Rob, If manual portal placement is useless for the majority of users, then a Quake3 BSP is the only way to go for the simple 1-click-compile-a-bsp-map-to-use thing that everybody can use. It's just an idea that's being presented. If anybody has ideas to make this better or comes up with an ever more clever way... share the idea.


marksibly(Posted 2003) [#72]
Todd,

a) Your algorithm is currently limited to 20 polygons.

b) Whether you like it or not, it works *because* the sectors are convex.

Which is not to say your not necessarily onto something here - keep hacking!


Pepsi(Posted 2003) [#73]
No no no ... you are missing the point.

That was just a simple illistration. If that was my design of my level, I could have around a 1000 polys in each room.

Grrr... let me draw up another illistration.


PS: Please, Kick the BSP stuff out the door on it's arse! I think I know what you are thinking with the 20 polygon thing. It's not that. <shivers>

PS: Gonna draw me room illistrations in circles this time...


LT(Posted 2003) [#74]
What !!?


Ross C(Posted 2003) [#75]
I know manual portal placement is not what this thread is really about, but surely each portal would be put in a type collection, and would have a list of all other portals it can see, again, entered by the user. I could be pre-calculated, or entered manually. Then when rendering the scene, you only render the room, which portals are on the 'can see list'.

Only thing is, you'd probably need two portals for every bit, as you would be going thru passageways bothways.


Pepsi(Posted 2003) [#76]
Ross, yes you could use 2 seperate portals at the same place defined with the same 2d convex/concave shape in 3d space. But, what you are suggesting then is to use portals as objects predefined to show other portals that they could possibly see from their position, not the camera position. It's a great idea and is what a trigger box solution is like, but it's no longer a portal because you are not looking through it in realtime to check to see if you can see other portals in a more accurate way.


Ross C(Posted 2003) [#77]
So why not use a trigger box solution. Then an entities which pass into a room that cannot be seen by the portal(or trigger box) carry a variable which holds the thier current room. They then get hidden if it is decided they are not in view, if you get what i mean.


Pepsi(Posted 2003) [#78]
I'm not saying a trigger box solution is bad at all. Me, I'm just portal crazy at the moment. :)


Ross C(Posted 2003) [#79]
hehe


Pepsi(Posted 2003) [#80]

- Red Letters = Room ( yellow lines )
- Green Letters = Portal (Blue Lines)

Here rooms B,C & E are convexed because all sides face each other. Rooms A & D are concaved because all sided do not face each other. With the portal system I'm talking about is:

- the level is sectioned out into seperate sections( entities of themselfs ) depending on where you want the portals to be in the level.
- portals either hides or shows each section (with EntityAlpha to keep collision) depending if you can see the portal in your camera view or not.
- No BSP alogrithm required.

Curently with maplet or gameSpace, I can create a single level. But I need to have it sectioned. So I CSG the sections of the level out. The doorway/window that leads into another section of the level is now replaced with a textured wall. I use this texture with a specific name to act as a portal. The specific name tells me what section the portal wil show or hide when I load the seperate .b3d sections of the level into my engine. My current demo will load levels like this.

What I'm working on is an editor to place portals manually without a bounding box thing to make overlaps a portal. The portal is created via vertices of the level geometry. This will allow me to create all different types of shapely portals at different angles then what you get with a bounding box overlap portal creation thingy.

How one get's a portal to see other portals or not is the creative part. The very basic portal shape itself that you place in a level editor should be the one thing thats common to all portal engines. The engine itself will figure how to work with these 2d shapes in 3D space themselfs.

My agorithm seems like a simple algorithm and should be fast enough to use. Thats what I'm doing now is implementing it. I'll will release the code and demo asap if it doesn't turn out to be bad. I have faith in it so far.


Pepsi(Posted 2003) [#81]
Oh, another thing is how this can work with the .b3d format. Hopefully I will be able to have all sections of a level in one .b3d including the portals.

in the .b3d format, I would have:
-node
|...- Section Mesh #1
|.....- node (child)
|.....- Portal Mesh
|.....- [plus more child nodes for more portal that see other sections of the level]
-node
|...- Section Mesh #2
|.....- node (child)
|.....- Portal Mesh
|.....- [plus more child nodes for more portal that see other sections of the level]
-node
|...- [ more section meshs here]

The node name can specify the mesh as a section of the level or it can specify it as a portal which will then also specify the section that portal shows or hides.

Hopefully I can EntityAlpha setions and use child entities as portals like this. I think Birdie with Quil3D is doing something like this.

note: Ofcoarse, we use the portal mesh also to see if we pass through it to see if we go into another section.


Pepsi(Posted 2003) [#82]
.


jfk EO-11110(Posted 2003) [#83]
Wow, I've just seen my radeon 9200 for 65 Dollars renders 390 Thousand Tris with 23 FPS...


Pepsi(Posted 2003) [#84]
.


Ross C(Posted 2003) [#85]
Cool, can't wait to see it. I'm gonna put a bit of work into my own idea. Well hardly my OWN idea but i'll give it a go anyway.

@Todd

The problem i can see you having, is if you have, say a corridor, which leads to another one. If you are looking into the corridor, the corridor further down from that should also be visiable. But i wanna see the dmeo :)


Pepsi(Posted 2003) [#86]
.


Pepsi(Posted 2003) [#87]
.


Ross C(Posted 2003) [#88]
Ok, now, something strange going on. Why does entities that are not in the camera viewport, but are hidden, give me a fps boost, whilst the entityalpha method, gives me no boost!

stange
any ideas?


fredborg(Posted 2003) [#89]
The Portal Master: That may be a sweet method for indoor FPS style environments. But let's face it, they are no fun compared to huge mixed indoor/outdoor scenarios.

Ross C: I think Blitz still needs to process 0 alpha entities, to check if they are in view. And if you have collisions going on, they are checked, while hidden entities are not.


Ross C(Posted 2003) [#90]
ok, cool. I've got no collisions going on and i'm manually checking to see if there in the camera viewport. Got it working better now. Missed out a command. :) thanks fredborg!


Pepsi(Posted 2003) [#91]
Fredborg, I can still use a portal on each side of a large sqaurish section of a terrain. I plan on making my outside terrain similuar to like the UT2003 demo's outdoor map.

My indoor/outdoor method will be smooth.


fredborg(Posted 2003) [#92]
Sweet :)


Mustang(Posted 2003) [#93]
Ok, now, something strange going on. Why does entities that are not in the camera viewport, but are hidden, give me a fps boost, whilst the entityalpha method, gives me no boost!


Do you have collisions?

If you wish to hide an entity so that it is no longer visible but still involved in collisions, then use EntityAlpha 0 instead. This will make an entity completely transparent.


http://www.blitzbasic.co.nz/b3ddocs/command.php?name=HideEntity&ref=3d_cat


Ruz(Posted 2003) [#94]
'the occlusion challenge'. is that anyhting like the Daz doorstep challenge .

Actually occlusion is a good name for washing powder.

'occludes the stains that other non bios leave visible'


JoshK(Posted 2003) [#95]
This map is 23,000 polys and 41,000 vertices.

It does contain geometry which your compiler will probably not like. However, Cartography Shop makes a distinction between "brush" type geometry, (convex and coplanar surfaces, with each face a separate surface), and arbitrary polygons, which is what all the pipes and details are. This distinction is lost upon export to .x format.

My BSP "compiler" (too soon to really call it that) just works with the brush geometry, and will use that to hide the detail meshes. I could provide you with code to actually check if a mesh follows the rules of geometry you need. If it does, include it in the BSP, if not, treat it like any other moving item in the map and just hide and show it based on the BSP walls.

This would allow your compiler to work with any mesh put out by any program. Bad geometry would only result in an object getting dismissed from the BSP tree.



Here is the same map, only without detail meshes. I can guarantee you that all geometry in this copy of the map will work with your compiler. Without detail meshes, the map is a mere 1600 polygons.




These super-high polygon maps would benefit greatly from BSP compilation. They could be compiled using only the "good" brush geometry, so that compiling a BSP wouldn't be any more comp[licated now than it would have been in the days of Half-Life. You just use the brush geometry to build the BSP tree, and hide and show the detail meshes depending on where they are relative to the BSP walls.


Ross C(Posted 2003) [#96]
@mustang i posted my post that i thought i posted before. :S
lol.

I didn't have any collisions. Very cool map there!


_PJ_(Posted 2003) [#97]


You don't want a routine that's too processor intensive as the speed you would gain from culling would be taken up by the time it takes to calculate the culling.




This is the main point. As really, occlusion is only about saving time, speed.

Another problem I envisage is 'cataloguing' all the objects so that yu can refer to them to know which one's are to be occluded or not.

I am still very much in favour of portals, even though they are a little out-dated now (ah dont kill me for saying that!) But they are a good balance (as per quoted statement).


poopla(Posted 2003) [#98]
I'm a bit out of touch here. Someone care to explain the terms used here: convex, coplanar, "brush geometry" etc.:).

<- dunce.


Rob(Posted 2003) [#99]
www.google.com :)


poopla(Posted 2003) [#100]
yeah, I looked ;). Looks like I'll be helping you lot now as well since I'll be needing this soon :).


poopla(Posted 2003) [#101]
RockStar, I would like to see that code. Can you post it please?


jfk EO-11110(Posted 2003) [#102]
Rob - sounds like you don't know it. Well, I don't know it - and I don't wonder! :P Well, convex is the opposite of concave, like when you say createcube() it's concave, and after a FlipMesh it's convex - or the other way :)


poopla(Posted 2003) [#103]
A convex set of polygons is one in which the polygons are both in front of one another(considering their normals) as far as I've read. Still havnt found a definition for brush geometry.


sswift(Posted 2003) [#104]
Any occlusion algorithm for today's 3D cards should probably work at the entity level, rather than the polygon level.

In other words, forget culling individual polygons. If any part of an entity is visible, all of it is visible.

This being the case, the primary problem of occlusion can be broken down into a simplified initital test for each entity.

Ie:
If it's bounding box is visible, then the entity may be visible. But if it is not visible, then the entity is definitely hidden.

If you did no other test, then this alone would provide a large speed increease.

So you need a fast way to determine if a bounding box is hidden behind arbitrary polygons in an arbitrary number of entities.

Which is not easy, but at least we have a clear idea of what needs to be done.


Another step one can take from here is to give all entities bounding boxes, and determine which boxes are visible based on these other bounding boxes. Of course this will not work if one's level is one large entity, but as I already said, you don't want to do polygon culling at the polygon level. So you want to split your level up into a bunch of entities for easier culling.


Of course testing if a box is hidden by other boxes is still not easy. But at least it will be a fast test.


Once you do this first test, you know which boxes are definitely visible, and thus which entities are definitely visible. So you can avoid doing more detailed checks on all those entities, but you still need to consider them when checking to see if they occlude other entities.

With this test you can also determine which SPECIFIC entities may be occluding a particular entitiy. So you only need to do the polygon perfect test on a known small set of entities.

And that's all that I have to say about this for now. I don't know what the best polygon perfect method to use would be, though "beam trees" looks interesting. May work well with this early-out test I described. It does NOT work well with large numbers of polygons though. So it's a matrer of how many polygons are too many and how many you have in each entitiy. I guess testing would have to be done.

type "beam tree" "abrash" and "quake" into google to get an article on Quake 1's rendering system whic describes beam trees and two optimizations for them which make the math fast.


Ross C(Posted 2003) [#105]
You could also have a flag for each entity, stating whether or not it is an occluder, if it isn't, it won't be considered in the tests.


LT(Posted 2003) [#106]
To answer Dev's terminology question...

A mesh is convex if all of its faces point outward and none of them, when extended to form infinite planes, intersect with any other faces in the mesh. Here's the test: if a 3d line can intersect the object in more than two places, it's not convex.

Of course, concave is the same thing but with all of the faces pointing inward.

Brush geometry refers to mesh details added on top of the occlusion geometry and usually parented to it as part of the visibility scheme. Think of it as furniture inside the room.


JoshK(Posted 2003) [#107]
-Convex does mean all the faces point outward.

-Concave is if any of the faces point inward, it doesn't have to be all of them, just one or more.

-Coplanar means that all vertices on a surface lie on the same plane.

-Brush geometry means that an object is convex, and is separated into coplanar faces.

Ross C, that is exactly how I do it, but I figured Mark was interested in compiling any mesh, not just output from CShop.

Dev, it would take me some time to put the functions into a useful routine, and I would only do it if Mark needed it.


LT(Posted 2003) [#108]
Semantics...

Yes, I suppose that is the standard definition for concave. My definition was in reference to objects that would be considered 100% concave - often used to define bounding volumes.

The requirement for brush geometry to be convex is specific to the technique used - no?


poopla(Posted 2003) [#109]
I'm going to be using both BSP, and manually placed bounding boxes/nodes(things that need polygonal detection within bounding boxes) to do culling I think. I don't see any reason to over complicate things. Most of it will be automatic anyhow. It might be a pain to have to manually place bounding boxes, but it only takes as long as you want it too. With a decent tool(Like the one I'm making for plasma atm), it wouldnt take long at all.


JoshK(Posted 2003) [#110]
Why manually place them? Just calculate the bounds of the mesh.


marksibly(Posted 2003) [#111]
Hi,


A friend of mine wrote this thesis on the use of portals in games



Very cool doc, Fredborg.

I like how he/she's taken such a different approach to cell/portal generation - a nice change from reading yet again how Quake was done!

Off to read up on 'Delauney Triangulation'...


marksibly(Posted 2003) [#112]
Yo, Rockstar!

Have you got B3D versions of those 2 models?

Also, any textures (just makes them more fun to play with)?


Bot Builder(Posted 2003) [#113]
that's the level in singularity 3.02, but I assume you already know that :) . My definition of convex(right out of a geometry textbook) is a shape in which no diagnols (lines between unconnected vertices) go outside the polygon. Of course, you still have to deffine what is inside and what's out.


JoshK(Posted 2003) [#114]
Here it is. (1.62 mb)

I don't have the original textures I used, but lightmaps are included.

"Brushes and meshes.b3d" contains the whole map. "Brushes only.b3d" only contains the brush geometry.

As a general rule, you might consider not including any named objects in the BSP, since they are probably special entities that will be moved, disappear, or are otherwise subject to change.


fredborg(Posted 2003) [#115]
I like how he/she's taken such a different approach to cell/portal generation - a nice change from reading yet again how Quake was done!

HE will be glad to hear :)


Boiled Sweets(Posted 2003) [#116]
Maybe I'm missing the point here but...

it seems to me that creating a 'world' from cubes is madness as for each cube you have 32 triangles. Wouldn't it be far quicker to use a surface/mesh comprising of just 2 triangles for a wall.

In other words instead of creating a 3d maze with say 10 * 10 * 10 cubes removing the cubes you want to be able to move through you create the walls using a 2 triangle mesh.

Am I making sense? Obviously that would cut down massively on the number of triangles poor Blitz would have to handle.


Ross C(Posted 2003) [#117]
12 tri's a cube? :P Apparently the number of poly doens't mater that much, it' the number of surfaces taht kills things and the number of different textures on screen.


Boiled Sweets(Posted 2003) [#118]
Well I figured its 4 tris per surface. Is it not? 2 tris for the front of side 1 and 2 fro the back of side 1. thats 4, then there are 6 sides to a cube so its 4 * 6 = 24 tris per cube (ok its not 32 but its not 12 either...)?

So lookkng at this thread it appears that Blitz3d is not really designed for 3d maze type games unless you can do some HEAVY DUTY occlusion/culling algorithms quickly.

Maybe I'll wait for BlitzMax to attempt my 3d maze.

What I would also like to know is if I had a cube and placed another cube beside it, how could I remove the 'walls'/surface joining the 2 cubes?


Ross C(Posted 2003) [#119]
12 tri's per cube, 2 for each face. The front of side 1 is the same tri's as the back of side 1.

You could figure out which cubes are beside each other and just strectch a cube for long walls. I think there is stuff in the code archives for that sort of stuff tho :)


sswift(Posted 2003) [#120]
"it seems to me that creating a 'world' from cubes is madness as for each cube you have 32 triangles. Wouldn't it be far quicker to use a surface/mesh comprising of just 2 triangles for a wall."

I don't know what CSHOP does with the cubes you create in it, but if it's not culling the hidden surfaces then NO THANKS. :-)

Btw, cubes only have 12 tris, not 32. 6 sides x 2.

Yes, ideally one would create a level which only has those tris that would actually be visible to the player. And all professional games that I know of construct levels in this manner.

In Angry Tanks, my own game, I have a level construction tool which allows you to place down 3d "tiles". Each tile is a 3D model, and all similar tiles are added to the same surface. Then you can push a button in the level editor to optimize the level, and it goes through and finds all triangles which have three vertices touching and deletes said triangles from the final mesh.

I posted this optimiztion function to the code archives a while back.

The trick though is that the function will only work if the triangles overlap. That means you have to specifically create the 3D models so that when they abut up against one another, the triangles match up. The cubes Blitz creates with the CreateCube comman on the other hand, (and probably the cylynders as well) do not have the triangles set up properly to do this. If you place two cubes end to end, and then look through from one side to the other, you'll see the diagonals cross instead of aligning. Ie, the front side diagonal goes top left to bottom right, and the back side goes top right to bottom left.

Solving this issue for just cubes with just two polygons per side would slow the algorithm down quite a bit, but solving for the general case... Well... I suppose if you created an algorithm which finds all coplanar tris and then makes a set of "faces" and then only compares one face to another then maybe you could handle less strictly constructed geometry, but you're still gonna have cases where you could say, place two cubes in a T contiguration over a third cube's front side and have a case where the third cube's front side is completely contained and should be deleted but is not detected. Such cases are hard to detect.

Optimization is already slow enough, taking about a second to complete for a small level which is just fast enough to use for doing it when the game is actually running when the level loads or possibly even when geometry changes if it's optimized more. But coming up with somethign that can handle any case would only be useful for precalculaton and saving the optimized level out, and that's not what I wanted for my game. :-) And it would be hard to accomplish. I don't recall ever seeing a tutorial on how to do that.

I basically just wanted a realtime optimization I could do when a level loads, which would remove the faces between cubes and floor tiles because those were creating tons of cracking artifacts. In my case, the number of polygons was so low that the speed of extra polygons in the level was not really an issue.


poopla(Posted 2003) [#121]
The editor im working on for plasma will have the option of contructing walls and such from said two triangle poly's. I guess I'll just call it "Strip" modelling. Har har har ;).


fredborg(Posted 2003) [#122]
Off to read up on 'Delauney Triangulation'...

Mark: Did you find anything? And if you did, can you give me a hint, I need it for this: ac3d to b3d converter topic thingy...


Techlord(Posted 2003) [#123]
I toyed with a precalculated occlusion technique similar to jfk's colour id system for 3DS levels. The system was fairly slow in processing vis data, however, the ultimate benefit to the system was that vis data collection was automated.

Using this technique I also realized that other processes could be precalculated during the VIS processing to include pathfinding tables and mesh swapping LOD. Obviously, these additional processes would impact VIS collection speed.

My "planned" solution to the speed issue was to developed a Networked Managment System that would distribute levels across a network of agent pcs for compilation. The NMS collecting the compiled data when complete. Many time consuming technologies use this form processing to speed up work.

No. 1, I'm not that excited about writting a game level editor when there are so many other tools that can do job. No. 2, manual placing portals does not sound like fun. No. 3, It is better to have it and not need it, then to need it and not have it.


JoshK(Posted 2003) [#124]
First, I want to define some terms.

Brush - The Quake geometry we are all familiar with, convex, coplanar, and capable of fast CSG operations.

Static mesh - High-poly meshes that are used to give insane detail to a map. These probably shouldn't be used to calculate occlusion!

Mark, you were right about no maps being complex enough to slow Blitz down, sort of. The polycount for the map I posted, without detail meshes, is 1900. That's without caulking (removal of any face that has a "caulk" texture). In my experience, caulking reduces a map's brush polycount by about 75%. So let's estimate the caulked map above to be about 500 polys. How big is a map going to get? A map ten times that size would be a meager 5000 polys to render, for the whole map. This is not even worth occluding. Just merge the whole thing into a single mesh and draw it. If a couple of Stroggs walk by, it's going to cost more polygons than that!

The static meshes, however, will kill performance. A map ten times the size of the one above would have a 350,000 static mesh polycount. So now we can come to a general rule: Brushes occlude static meshes. And that's it. Brushes never get hidden, and static meshes never occlude anything. Let's just use the bounding boxes of the static meshes to test occlusion. If anything, doing that will err on the side of caution, and never result in any visible mistakes. Suddenly our job just got really easy. We have a bunch of boxes to hide, and a bunch of flat walls to hide them with.

Mark, you saw my CSG occlusion demo last Spring, where I calculated occlusion by performing CSG on the outlines of objects, relative to the camera. If the same thing were done, using brush walls as the occluding object, and just using a bounding box to test occlusion of a static mesh, large levels with insane detail would work.

I'll have a large UT2003-detail map running in a week. The map might be kind of repetitive, but will show the technique.


Rob(Posted 2003) [#125]
Mark is clearly *wrong* because he hasn't taken into account what we might achieve.

Consider a node
500 polys of high detail.
x2 or 3 when you multitexture enough times.
Add cubemaps.
Add bumpmaps.

only occlusion will save the day. Especially if the level has 300 such chunks.


JoshK(Posted 2003) [#126]
I don't think he ever literally meant that we couldn't slow it down. I think he just meant the maps he was testing with. I shouldn't have said it like that.


Rob(Posted 2003) [#127]
Yeah, I didn't think either - just suprised he'd say that, thats all.

When it comes to it, HW occlusion is probably gonna be the the all-round solution to preventing overdraw.


Isaac P(Posted 2003) [#128]
*bump*


AbbaRue(Posted 2004) [#129]
I finally know what Occlusion means, and what it intails.
Quit the subject.
I'm glad I dug up this old post from last year.
A little dusty though.
Any new thoughts on the subject?
I think LOD is really the best way to go, at this time.
And that is were I am at now.


Michael Reitzenstein(Posted 2004) [#130]
Quit the subject.

Erm... says the person dragging the post up?

I think LOD is really the best way to go, at this time.

LOD is suited to models and occlusion is suited to indoor environments, they are two vastyl different things.


Michael Reitzenstein(Posted 2004) [#131]
I finally know what Occlusion means, and what it intails.
Quit the subject.
I'm glad I dug up this old post from last year.
A little dusty though.
Any new thoughts on the subject?

You are kidding, right?


marksibly(Posted 2004) [#132]
Its still a very important topic!

Ideas?!?!?


Michael Reitzenstein(Posted 2004) [#133]
Its still a very important topic!

It is, but he posted:

I finally know what Occlusion means, and what it intails.
Quit the subject.
I think LOD is really the best way to go, at this time.



In a dead topic months old, then edited it at my response.

As far as occlusion goes I think the request I posted here could really help with instances where one object occludes another (though it would be entirely useless for self-occluding objects), wouldn't it be worth implementing as it is essentially 'free'?


AbbaRue(Posted 2004) [#134]
From what I can figure, the time needed to check which objects in a scene Occlude others,
the scene can be rendered as is. The clock cycles used to check a 3D object are the same
clock Cycles needed to render the object.
Todays GPU's are faster working with 3D objects then CPU's.
That is why I mentioned LOD as a better method of doing things.
I wasn't refering to indoor scenes.
I asked a question earlier today, concerning an outdoor scene, and was pointed towards Occlusion.
That is why I looked it up in the first place.
When I walked up to a hill in my world, I noticed that the tris. value remained the same.
So I asked if there was a way to tell blitz not to render past the hill.
That was when I was told about Occlusion.
I think for outdoor scenes LOD is still the best way to go, because of the distance and complexity.

For indoor areas, I could see Occlusion or something simular as priceless,
and it should be quite easy to implament. I can see some means of doing it already.
But that is not were I'm at right now.
When I get to the indoor part of my game I will be giving it alot of thought.
But right now I am working on the outdoor part of my game.


Michael Reitzenstein(Posted 2004) [#135]
Ah, I see. This whole debate, though, (as I remember it) is about occlusion indoors. You can't simply LOD indoor environments (though you can LOD the objects in them).

You are dead right about GPUs being very fast nowdays and that occlusion isn't necessary for many scenes. What can be done, though, is to create a very 'loose' occlusion algorithm that occludes huge chunks and sacrifices quality for speed (but it *always* renders when in doubt), which won't use much CPU time but will help a hell of a lot on the GPU (though the GPU will still be under significantly more strain than it would be with a Quake 1,2 & 3 style occlusion system) in huge indoor worlds, and I think that's the way forward.

It's easy to say that, but coding it (especially when setting it up in an automated fashion) is another thing! :)


jfk EO-11110(Posted 2004) [#136]
I still got this Idea for an Occlusion on a very low machine level:

Render from near to far, and as soon as all Pixels of the Screen are drawn (Count Pixels), skip Rendering. I have no idea if this is possible with DirectX, and it might need to use non-intersecting Polygons. But it would be pretty fast.

I wanted to use this for a software 3d engine some time ago, but never finished it.


Michael Reitzenstein(Posted 2004) [#137]
You can *kind of* do that, but you can't tell the 3D card to skip rendering when all the pixels are drawn.

My request for EntityOrder with ZBuffer was to do something like this - it's faster to render an object in front then an object behind than it is to render an object behind then an object that is in front. It will still require some CPU time to order them, though - and it's no replacement for a 'real' occlusion system.


AntonyWells(Posted 2004) [#138]

Its still a very important topic!

Ideas?!?!?



Depends on whether you mean from within Blitz3D, or within the engine it's self.

From within Blitz3D, there's a method I developed, though slow, so the actual mesh compiler is best wrote in C++,

compile stage,
go through each poly in the scene, and from it's centre, cast a linepick to the centre of every other poly in the scene. If it can 'see' the poly, add it to it's seen.poly pool, if not, add it to it's unseen poly pool.
One way to speed this up is to create a 3rd universal list, that each poly is added to as it's check, that way when you check a poly, check if it's cross-checked in this universal list, if so, you can safely assume if it can see or not, the ones that references, and so on, end up with 50% less linepicks.

Anyway, you have this list of every poly visible(For memory reasons, it would probably make sense to make it every 4 or 10 polys, or better still polyInc#=numPolys/maxNumberTotal (maxNumberTotal being worked out at runtime depending on quality settings etc.) now, due to blitz3d's limited access to internal rendering, we have to create meshes with overlap,
firstly, group every poly that can each other in unique meshes, so say poly a can see poly b and poly d. that means b can a, but whatever, at any point on this section it's worth assuming they can all see each other. This becomes a surface.A long corridor for example would become one mesh.
For accuracy, a further check could be to use a threashold for max amount of non-visible pairs before it's disqualified and is apart of the next surface created.But it's still added to the current mesh, because it can be seen from some points, but also the next mesh, so this is the overlap, so we can safely changes visible meshes.

Now when we create the next mesh, we link it the previous, and link the previous to the next, do this by generating a bounding box.


Then, you have a bunch of meshes totally independent of scale/size, and again totally oblivious to the shape of the 3d world.

Save these meshes, out, along with list of links, and the bounding boxes.(The meshes are linked to the bounding boxes too)
Then in game, we just check what bounding boxes we're in, remembering they can overlap safely, and turn the alpha up on those meshes, turn it off those we arn't in.

ultra fast, just ultra expensive to compile the meshes..but i'm sure it can be improved further.

The biggest drawback of this is down to way you handle surfaces now. It appears you bind the textures of every surface, even if the surface shares the same texture combos.
Imo you should group off each surface in memory that share the same texture combo. that way it surface limit problems would disappear, and it would be the much better(if it a little harder to understand for newbs) limited by unique texture combos/blend modes.
-

if on the other hand you're talking internal, I think streaming is the way to go for huge worlds. just look at gta3 on the xbox and then compare it to Deus ex 2 on the xbox, which uses occlusion etc. One has huge open worlds, the other has claustophobic levels riddled in a load fest of a mess.

i.e build up a mesh from a set point of origin, and then as it transforms away from the current camera, stream in a certain set of polys and stream out just as many. order it in memory in terms of Z, so you can quickly throw out a bunch of polys without any expensive maths, and check for the closet x or the first ones to fall within a radius, and throw those out. Do the same for streaming in, but save them in order of Z.


Rob(Posted 2004) [#139]
Count pixels doesn't work due to alpha pixels. Overdraw is a necessary evil.


Zmatrix(Posted 2004) [#140]
doesnt Bsp /vis create alot of surfaces?
3dgs for example(maybe the engine is just broken)..i had a small 3000 poly level made in Cshop and exported to .map , in 3dgs it was over 8000 surfaces after compile.
in b3d it was 6.
3dgs ran about 30fps
while in b3d about 225fps


No doubt b3d is fast,not just in 3d but also in code execution.
http://www.glbasic.com/files/mandelbrot.zip
A fractal drawing test/compare for blitz3d,db,dbpro,glbasic,Vb6,Vc++6
on this crappy system, 950 duron,256megs radeon7200 32(horribly slow pci version) heh

vc++ : 260 kpix/sec
blitz3d 1.64 : 225 kpix/sec
vb6: 200 kpix/sec
glbasic 180 kpix/sec
db: started drawing but woulda taken about 10 min to finish the first screen.
dbpro : never even saw it start,,said 0 kpix/sec
mayeb someone with B+ can port it?...i would like to see the score.
.............

Blitz3d using Retained mode?? I dont think so..heh
Alpha doesnt work in retained mode,you cant multitexture and theres no hardwaret&l support..best youll get even out of even a geforcefx or radeon9800 in retained mode is about 5 million tris per second.(give or take with Cpu speed)
3drad is retained mode and ive seen 4million tris per second (single surface) with my radeon 8500.



Zmatrix


Shambler(Posted 2004) [#141]
No need to count pixels, rendering objects in near to far order will give a good performance increase as is.

Plus if you count pixels this won't work if your meshes are concave or overlap.

Bsp splits polygons if they straddle the current partitioning plane so creating an extra poly each time.

There's no holy grail of culling methods because they are so reliant upon the type of geometry you have.

That doesn't mean that there isn't a general solution, just there is no 'best' solution.

One general way that would work for indoor and outdoor scenery would be to maintain your own z-buffer in software, draw near to far and use this to decide what needs drawing.

On the other hand you may find that the gpu on the graphics card does a faster job for free! =)


AbbaRue(Posted 2004) [#142]
I could see using a 2D map of an indoor scene, to seperate rooms and then just render the present room you are in as a seperate mesh object. Then use more then one camera to do Windows and doors, before you enter the other rooms. Kind of like a mirror idea, only it's a window or door, not a mirror.
Then you don't need to worry about Occlusions.
I don't understand why this wouldn't work for any indoor scene. Any examples of were it wouldn't, before I write code and then find it falls short.


TeraBit(Posted 2004) [#143]
Excuse my ignorance, but what information does a BSP hold? I know that it is a list of what triangles are visible, but how is that held. As a spatial grid with a list of tris attached to each area?

I have a method for determining what triangles are visible that I developed for the Projection Painting in Tattoo, and I was wondering if it would work for visibility compling?


Rob(Posted 2004) [#144]
Part of the problem isn't actually how the data is held, but getting the data in the first place. Even a lose tree of data will do us.

But how to compute visibility from where you are?

IMHO the best occlusion would be on a per-entity basis. This would work as Hiding/showing large batches of polys is beneficial in todays hardware.

The problem is scanning the scene and building a list of visible objects for each x,y,z map grid position. I have done something like this with linepick, but compilation is painfully slow.

Optionally, the engine could manage and split the level into chunks.


TeraBit(Posted 2004) [#145]
The problem is scanning the scene and building a list of visible objects for each x,y,z map grid position. I have done something like this with linepick, but compilation is painfully slow.


I see. How slow is slow? And what resolution is the vis data held at. i.e. 'at each blitz unit?' or some other arbitraty resolution?

The method I mentioned would give you a list of triangles and surfaces visible from any arbitrary point. It's fast, but not instant. Should be much faster than vanilla blitz linepicks though. Do you have an blitz code example?


jfk EO-11110(Posted 2004) [#146]
Two problems arise: Illegal Geometry and Compilation time. You have to define a resolution for the grid of points you'll use to check for visibility. The more points it has, the longer the VIS calculation and the longer the ingame hiding/showing. Depending on this Resolution there is a certain kind of ILLEGAL GEOMETRY, which could be visible from one footstep byside the checked point, but remains invisible from the point. So you cannot simply use any beloved Map or Model. An object must be bigger than the distance between two points, watched FROM ALL SIDES.

Checking a level with say 50*50*50 Positions can easily take up to several hours using Linepicks. So slow is slow. ALthough it is a precalculation, it's uselessy painful.

Some time ago I have released an other method that is indeed much faster. It is using a "Identify Color by Uniqueness" Method (ICU). Basicly every Surface is using plain Color and no texture. Every Surface will have it's own, unique Color.

Now a camera is positioned at every point, take 6 Renders like in cubic enviroment mapping. Now use readpixelfast to check for the colors on the rendered images. It's even possible to use the color right as the index of a blitzarray that acts as flags for visibility. For 16 Bit color Resolution the values might have to be shifted some way to work correctly, but in 24 or 32 Bit it's really simple.

A working example can be found here:
http://www.blitzbasic.com/codearcs/codearcs.php?code=341
(note: use it to see the ICU method in action, other parts might be spaghetti to useless)

There is a further problem: if the initial Map has let's say 20 surfaces, you will need to fragment the Map Mesh into clusters to be able to hide the occluded parts. Therefore each cluster will contain its own copies of the surfaces it uses. This means you will now have about ten times more surfaces in the videoram, depending on the leveldesign.


TeraBit(Posted 2004) [#147]
Hi jfk,

Thanks for the source. I ran a compile on my machine and it took 6 minutes to complete.

After some messing around, I managed to get it down to 16 seconds (22.5 x faster). That's about as fast as I can see it going with either system. The lockedpixels stuff of Blitzplus would help here I think!

[edit] got it down to 12 secs. :)


TeraBit(Posted 2004) [#148]
So is that all that is required, or does it need to be done on a Triangle by Triangle basis?

How would you go about hiding and showing quickly the tris that are hidden / visible.


jfk EO-11110(Posted 2004) [#149]
I'd keep an array containing flags which show if a surface is hidden or not. Then I would only hide/unhide those which have to be altered. That's fast.

I have no idea for a By-Triangle solution. I think it would be pretty slow to assemble a meshs visible Tris each loop.
I also think you would end up with a too high number of surfaces if you would make each Triangle a hideable Surface or child mesh.

And whatever you do, a map must be designed for occlusion, otherwise it will not be much faster. (no big rooms with high detail)

BTW, Lee, how did you reduce the time to 12 secs?!?


TeraBit(Posted 2004) [#150]
In this case, looking at what was taking the time, I set up multiple cameras looking around in four directions. You can then set their viewports at different on screen positions. That way you have one renderworld with four faces. (you can get a better verticle field of view with a lower camerazoom which eliminates much of the look up and down)

It was also taking a long time to lock and unlock the buffer, so having to do it less often by having 4 in 1 renders sped thing up a bit.

Locking and reading the backbuffer/frontbuffer seems to be very time consuming, so I copyrected the composite four faces to a created texture which has the +256 flag to lock it in VRAM.

From that created texture reading is much faster (really much faster). But it would be better still with locked pixel stuff.

My plan ultimately would be to render two positions in one frame by using up the remainder of the texture space in a 1024x1024 frame with another camera set. That would lower the overhead on the copyrect to about half. So about 9 seconds would be about right.

I could get it down to between 5-7 secs if I could render directly to a texture!


jfk EO-11110(Posted 2004) [#151]
thanks! pretty sly. well, i think when I wrote it there was no 256 flag :) . Honestly I had no idea reading the backbuffer is slower than reading from a texture.


Techlord(Posted 2004) [#152]
I devised a color id VIS compilation system similar to jfk's. In fact, our algorithms are practically identical:

;frankie t's identify color by uniqueness (icu) algorithm
array vis(n,n,n)
global worldscale#=.1

loadanimmesh levelmap.b3d

for each childmesh within mesh
	apply unique color equal to childmesh index
	set childmesh attributes to fullbright, flat shading, disable fog, disable backface culling, no texture
next

repeat 
	increment camera through level map geometry on x,y,z planes
	cameraPosition camera,{camera.increment * worldscale#}
	vis(camera.increment.x%,camera.increment.y%,camera.increment.z%)=create bank
	for cameraAngles% = 1 to 6;(up,down,left,right,front,behind)
		cameraAngle camera
		snapshot=cameraSnapshot(camera)
		for each snapshot
			for x = 0 to snapshotWidth
				for y = 0 to snapshotHeight
					color%=readpixel(snapshot,x,y)
					if color>0 and not on vis(camera.increment.x%,camera.increment.y%,camera.increment.z%)
						poke(vis(camera.increment.x%,camera.increment.y%,camera.increment.z%),++,color)
					endif					
				next
			next	
		next
	next
until camera completes level



I have found "Identify Color by Uniqueness" method to be very flexible, easy to implement, and faster than most. It works for convex/concave geometry. You can also precalcuate mesh-switching LOD with it.

The process is extremly slow on older machines. To solve this problem, I was rewriting the compiler to split the compilation task among multiple agent PCs. This multi-processing approach is commonly taken with other task that have long compilation times such as cgi.

However, I could live with TeraBit's compilation times. So I guess all I need to do is get a faster machine.


Rob(Posted 2004) [#153]
Terabit's optimisation has allowed a far greater degree of accuracy, so perhaps this technique is feasible after all.

Nice work lee!

Mark, do you still think render to texture is out of the question? If so, why?


Techlord(Posted 2004) [#154]
Interestingly. I too use multi-viewport technique parented to a pivot. But I only render to the backbuffer (no flip to the frontbuffer). I also increment the read pixel x,y by 2. Hmm, the copyrect texture VRAM method sounds very good.

Overall, the ICU Algorithm is my VIS method of choice. It could very well, become a standard.


Pepsi(Posted 2004) [#155]
Manual Portal placement will become the standard... no doubt! :P

Really though, I'm getting there and I will show all the greatness of Portals!

I thought Rob might like some hype there... :D


Rob(Posted 2004) [#156]
I also increment the read pixel x,y by 2
pointless optimisation. Instead, render smaller!

Hype? even the matrix couldn't touch wells for that.
Anyway, I like hype, just not antony wells hype.

I believe I will use Lee's method. I did this form of occlusion a long time ago, probably around same time as jfk.

An optimisation I did further was to replace all entities with bounding boxes. This sped up the render portion significantly for further speed gains. The loss in accuracy was minimal. The slowest part isn't rendering, it's scanning.


TeraBit(Posted 2004) [#157]
The copyrect to the texture is really slowing things down (although not as much are reading from the backbuffer!)

You can use the same method for Triangles too. Simply unweld the geometry and use vertex colours to colour the individual triangles the colour ID.


Rob(Posted 2004) [#158]
Do you honestly consider this to be a good method?


TeraBit(Posted 2004) [#159]
You mean for triangles? Well I've been using it in Tattoo since it got projection painting (otherwise reprojection would take about four minutes instead of a few seconds).

You can tell exactly what triangles are visible at any location even if they are partially occluded. It's never given me any problems and vertex colours render very quickly.

I tried to post my generic function to do the job, but my work net connection won't let me post anything over about six lines long.


TeraBit(Posted 2004) [#160]
On the other hand, if you mean do I think that culling individual triangles is a good method, then I would say no. Not worth the effort. I don't believe that it makes sense to only render a few hundred polys over a few thousand considering the processing overhead involved.

Basing the occlusion on entities is a much neater, more controlled solution, so jfk's CSP stuff does the job nicely for most things.

That said, some changes to the entity system would make this kind of system easier. Perhaps some kind of grouping. So you could define a list of polygons as belonging to a group (although with less overhead than a separate entity, since they would not have rotation, scale and other entity stuff to worry about.)

Grp = CreateGroup()
AddGroupTriangle Grp,Surface,Tri

HideGroup grp ; removes from rendering pipeline
ShowGroup grp ; shows in rendering

Thoughts?


jfk EO-11110(Posted 2004) [#161]
if you mean do I think that culling individual triangles is a good method,


I think someone really should make a benchmark test. I suggest to try the following: Based on Triangle Visibility Information a prototype Mesh that contains visible Triangles only is constructed each frame. It may take too long to assemble it, then again it's only a few hundred tris. As far as I know, MasterBeakers Z_Sorting Algorithm for alphaed meshes used to reconstruct the mesh each frame too.

Maybe it would work this way: keep a hidden copy of the entire Map in memory and clone the visible trianlges only, just to render and remove them again.

I see more reasons to use VIS data: Currently I use Linepicks to determine if a Lightbulb is visible and needs a flare. Those picks are real slow, my app would profit a lot if it could use VIS information to show/hide flares.

Somebody should find a clever way to save the VIS Data in a Ram-saving way. Even if it only contains the visible Tris for each point, its size will easily be 20 Megs. So maybe some kind of compression is required.


TeraBit(Posted 2004) [#162]
To a certain extent, that was the idea behind groups. That way, you could simply list the groups that were visible from each point.

Reconstructing a large mesh in real time in blitz would very likely be time consuming. A built in system would fare better, but may be overkill in the current GFX climate. Pehaps since the way forward is bigger and heavier graphics cards, a simple group based occlusion would do what we need. There may be some dynamic way of setting up optimal groups in code using a similar system to the CSP.


TeraBit(Posted 2004) [#163]
Everyone lost interest in this then?


Rob(Posted 2004) [#164]
no, not at all. I think that render to texture needs to be put into blitz. It's foolish to leave it out. Put it in and we will simply benchmark it's performance and choose to use it or not.

As programmers give us the choice, I say.

Lee, why not put your point across to Mark about render to texture? so many effects and techniques use it now it seems foolish not impliment it at speed.

Render to texture could also preserve masking.


jfk EO-11110(Posted 2004) [#165]
I think I'm gonna try to reconstruct the mesh using the visible tris only. Since this isn't a large mesh, especially if a lowpoly map design is used. Maybe it's gonna be 300 tris or something. I think it's worth a try.

For this i'd check all surfaces of the hidden original map. If anyone of the tris of a surface is visible, I'd add that brush/surface to the visible mesh. So I'd probably have only 5 surfaces to render, while the whole level uses hundreds of surfaces. In the same time the whole map can be real big. I just need to find a way to ship around the 64k Vertices limit. How exactly does this limit work, is it possible to use 64k Vertices per Surface/child, or does it count for the entire top level entity?

However, if I can make it compile it 12 secs instead of 6 minutes, it's much easier to develop.


Techlord(Posted 2004) [#166]
This was very good discussion on VIS and occlusion. Could not resist reviving.


Pepsi(Posted 2004) [#167]
.


poopla(Posted 2004) [#168]
heres a VERY simple way to do highly efficient culling.

All ya do, is break the map up into 1 x 1 grids. Check every triangle in the map for visiblity, using normals, and a visibility routine. Store all visible tris for each cube in the 3d grid. At runtime, clear all tris, and only build the visible ones.


Techlord(Posted 2004) [#169]
IMHO destroying and building the tris at a runtime could prove to be very inefficient.


RemiD(Posted 2016) [#170]
<-!necro post!->

I have thought of an easy to implement system to determine the non visible surfaces and the visible surfaces and show only the visible surfaces before rendering :

-when you create your map, make sure the static entities are divided in different meshes (or copies of a mesh) depending on their position and proximity with others static surfaces and shared or notshared materials with others near static surfaces. (of course it is better to combine surfaces which are near and which share the same material (you can also combine different textures which have the same material into one and then combine the corresponding surfaces into one))

-make sure the turningmoving entities have each one mesh (or copy of a mesh)

-for each static entity, and for each turningmoving entity, create one very low details mesh (the less tris possible, just to have an approximate shape) and one high details mesh (as much tris as necessary)

-for each static entity, and for each turningmonving entity, create a root and a very low details mesh (to determine the visible/notvisible surfaces) and a high details mesh (to display the result to player)
Color each very low detail mesh with a different color between 0,0,0 and 255,255,255 (you must use 32bits graphics) and set it to fullbright.

-before rendering the scene with the high details meshes, hide all high details meshes, show all very low details meshes, set the cameraviewport to 10% of its normal size (for example 102*76 for 1024*768), render the scene, analyze the pixels of the image, and depending on the colors found, you know which entities are not visible and must stay hidden and which entities are visible and must be shown.

-hide all very low details meshes

-show the necessary high details meshes

-render the scene

That's all, it can be very useful for an outdoor map with big obstacles like buildings or deeps hills or mountains which can sometimes mask others smaller shapes...
But for an indoor map, it is probably better to use a portals system (one room connected to one or several others rooms by one or several passages, before rendering hide all rooms (and the statics in each room) and depending on in which room player is, and depending on the passages from this room to the others room, show the appropriate rooms (and statics in these rooms))...


Kryzon(Posted 2016) [#171]
Take a look at this:
http://http.developer.nvidia.com/GPUGems/gpugems_ch29.html


Rick Nasher(Posted 2016) [#172]
Much like a particle system for grass and stuff?


RemiD(Posted 2016) [#173]

Much like a particle system for grass and stuff?


No ! :P

You may know what the camera fov (field of view) is, all meshes in the camera fov will be considered for the rendering, all meshes outside the camera fov will not be considered for the rendering.
However sometimes some meshes considered for the rendering are not visible to the player because there are big meshes in front of the camera (so a big mesh masks one or several smaller meshes behind it).
So the "visible/notvisible surfaces determination" is the process to determine which surfaces are visible and which surface are not visible (because a big surface may be in front of the camera and some smaller surfaces are not visible). This process is usually done with bounding boxes or very low details meshes.
Then only the necessary high details meshes are shown and rendered.
This decreases the time it takes to render a scene. But it is useful only with some kinds of scenes (outdoor more than indoor)