Quake 1 BSP loading

BlitzMax Forums/BlitzMax Programming/Quake 1 BSP loading

Hezkore(Posted 2016) [#1]
Seeing as how loading Quake .MAP files turned out to be really hard: http://www.blitzbasic.com/Community/posts.php?topic=106944
I decided to instead load the .BSP files.

I'm having some problems generating the vertices though.
The information seems to read correctly from the BSP but generating a mesh based on vertex data just gives me messed up mesh.
I'm guessing it's because the Vertex list order isn't correct and I'm supposed to use "Edges" and "Faces" but I'm having trouble understanding it...
This is what I've got so far:


If you look to the right you'll see this mess


The specs can be found at: http://www.gamers.org/dEngine/quake/spec/quake-spec34/qkspec_4.htm
BSP file from Quake 1: https://dl.dropboxusercontent.com/u/2842751/DM1.BSP


JoshK(Posted 2016) [#2]
That particular map format uses plane equations. I think it provides three vertices you construct a plane from. You then have to calculate the convex volume defined by that list of planes. It's complicated and prone to errors, which is why maps can sometimes become messed up in the original tools.


Kryzon(Posted 2016) [#3]
Hi Hekzore, this is where looking at previous work can help.
I searched for "BSP loader github" on Google and found this:
https://github.com/mikezila/uQuake1

According to the specification, vertices form edges, the edges form faces.
To associate a face with its vertices you need to look at the edges. Some faces may use the same edge index (meaning, they have an edge in common), so the order of the vertices in an edge is given by the sign of the edge index in the global edge list.

Interesting points in that importer:
- https://github.com/mikezila/uQuake1/blob/master/Assets/Scripts/uQuake1/BSP29map.cs#L21 (Loading data)
- https://github.com/mikezila/uQuake1/blob/master/Assets/Scripts/uQuake1/GenerateMap.cs#L65 (Making the mesh)


Hezkore(Posted 2016) [#4]
Yeah thanks Kyrzon, that helped a bit.

Right now the problem is that I don't understand how Faces specifies the Edges it uses.
My understanding (based on this: qkspec_4.htm) is that Faces uses its ledge_id to specify the starting Edge I should read and then continue forward to ledge_id + ledge_num.
But there are a total of 5688 Edges in the BSP I posted above, and if I go through the Faces that way they request Edges as high as 11370!
Here's an example (Run in debug mode and check console)



Here's a version that doesn't use MiniB3D so you can try it without any extra modules.
Just paste this into BMax and run in debug: http://pastebin.com/raw/5fTsHxvi

BSP file from Quake 1: https://dl.dropboxusercontent.com/u/2842751/DM1.BSP


Hezkore(Posted 2016) [#5]
Okay I've had some progress!
I figured out that there's a LEdge type too that tells me what Edge to use.
I've gotten to the point where I can actually read Vertex data now, but I'm still really confused as to why there's two Vertices in an Edge and how to generate the mesh.

Here's the new code:


The interesting part is at "'Debug make a mesh"


Kryzon(Posted 2016) [#6]
An edge is a segment of a 3D line. That segment is formed by two distinct 3D points, the vertices. All edges have two vertices.

There are two lists of edges in that BSP file:
- One list, "edge", defines all unique edges in the scene.
- Another list, "ledge", defines all faces by specifying consecutive edge indexes from the "edge" list that form each face (a face can be a triangle, a quadrilateral, a pentagon etc.).

I didn't run this, but maybe this helps (EDIT: corrected something):


This all falls apart when you add in texturing though. This BSP format is supposed to work with that WAD pack that contains all textures.
How do you plan on doing that?
Maybe working with DeleD CE should be better, it exports to B3D format directly with texturing and it's a map editor similar to TrenchBroom. The model would come cut up into surfaces and ready for use.


Hezkore(Posted 2016) [#7]
I managed to make some good progress.
I've gotten the geometry pretty much perfect.
I can load the palettes from Quake 1 (I'll just include one in the module by default).
I can also get the textures and apply them to the geometry!
But there are some problems with the way I'm drawing the textures, so some are still a bit wrong.
I'm going to work on lightmaps next.

@Kryzon Actually the WAD files are only for editors!
The textures are stored directly in the BSP for each map (what a waste)
Which means I only had to read the Quake 1 LMP palette file to get the colors correct.

Anyways, here's a screenshot of the progress so far:



Naughtical(Posted 2016) [#8]
Impressive! Can't wait to see more!


Hardcoal(Posted 2016) [#9]
Are u making a map editor?


Hezkore(Posted 2016) [#10]
@adiaden Thanks!
I've got the textures fixed now so they're all correctly aligned.
Liquid textures such as water and teleporters sort of work now, not as fancy as in Quake but they move!
Transparent parts also works, not sure if original Quake really supported that, but I feel like water and lava should be a little transparent.
I'll tackle Lightmaps next, should do a lot to the overall atmosphere.

@Hardcoal Nah.
I'm only loading .BSP files, which Quake 1, 2 and 3 uses.
It won't load the Quake 3 .BSP files but it does load Quake 1 ones, not sure about Quake 2 but I doubt it.
There are already plenty of editors for Quake 1!
I personally love how TrenchBroom 2 works, it's super easy and fun.


Kryzon(Posted 2016) [#11]
I remember that location, level 1.
I love the steampunk, grungy gothic art-direction in Quake, but I also remember hating to have to kill the dogs; would avoid doing that when I could.


Hardcoal(Posted 2016) [#12]
You still didn't say why you are loading them or did i miss that.

I used to make maps with quark army knife


Hezkore(Posted 2016) [#13]
@Hardcoal No I guess I never said why heh.
No reason really, I just figured it'd be good to have.
As I said, I love TrenchBroom 2.x (and Quake!) and if I ever decide to make a 3D game I could easily make maps with it and load them with this code.
I'm planning on releasing this as a module for everyone to use and modify.

@Kryzon Yeah I love the style too!
The lightmaps aren't the best in Quake 1, so I think a lot of the style comes from its limitations heh.


Hardcoal(Posted 2016) [#14]
thats genius ... I actually might need it sir..
I have the same ISsue..
I can make great quake 2 maps.. /quake1
and I can use it on my Game Editor.


Hezkore(Posted 2016) [#15]
I've managed to get some lightmaps loaded.
The UV for them is still a bit wonky which creates some ugly seams and odd parts.
I'm hoping to solve that once I merge all the tiny lightmap images Quake uses, into one big lightmap image that I can throw onto the entire mesh.
Having all of these small lightmap images means lots of surfaces too which means it runs at a terrible speed right now heh.




Hardcoal(Posted 2016) [#16]
Great job mr..
You are very fast and productive..


Hezkore(Posted 2016) [#17]
Alright so I've managed to get basic scrolling skybox support in.
Animated textures now also works correctly.
Lightmaps.. sort of works, it's kicking my ass actually.
At some places it looks great, but other, seemingly random parts are weird.
I'm guessing it's the UV coords, but the documents aren't even near correct when it comes to the lightmaps.
I've had a friend look over the Quake and QBSP code and we've gotten some extra information from that, but they're still not perfect.
Lightmaps at least one poorly put together image now, instead of many tiny pictures.
So performance is back.






Naughtical(Posted 2016) [#18]
Looks great!


AdamStrange(Posted 2016) [#19]
yep. it really looks good


Hezkore(Posted 2016) [#20]
Thanks everyone!

I'm still trying to optimize and load everything from the BSP.
Just got entities working, so now it's possible to get information such as weapon and player spawns, triggers, doors, buttons etc.
It doesn't mean that stuff moves around in the level, it just means we can read the data for such things and possibly add support for stuff like that later.
Though that's more of a "game" thing and not something the BSP loader should really handle.

I've also added support for some Quake 1 community tricks and modern stuff.
For example it's possible to have textures with transparent parts (masked) which original Quake 1 doesn't support.
Also, some REALLY huge maps are possible to load, much bigger than Quake 1 normally loads.

I could do lots more if BlitzMax had a proper 3D engine.
MiniB3D is very limited.

If someone knows how to load MDL files with animations (Basically the same as MD2 files) I could get those into the map too.

Here's a random screenshot from when I was testing GL_NEAREST on textures.
Notice the vines are transparent with lightmaps.




Kryzon(Posted 2016) [#21]
BlitzMax-Assimp has most of the work done for you. It can load MDL and several other formats, but you need to add animation support:
https://github.com/Difference/blitzmax-assimp

It has some code to convert from an assimp mesh to MiniB3D, but the module did not implement the animation features. It seems abandoned.


Hezkore(Posted 2016) [#22]
@Kryzon Yeah I've attempted to use Assimp before but it's such a mess.
I know MiniB3D Extended has animated MD2 support, maybe I can modify that.


Anyways, I've put the code on BitBucket along with a test BSP and example:
https://bitbucket.org/Hezkore/bsp-loader

The lightmap UV is still wrong.
And skyboxes don't work like in Quake 1.
So don't expect things to look right.

MiniB3D is, as I've said, very limited.
I'd like to use the stencil buffer and stencil in the sky, but I can't with MiniB3D.
I've already had to hack MiniB3D to get GL_NEAREST filtering on the textures.
That means that you guys will get some blurry textures.
If you want to use GL_NEAREST for sharp textures you can apply this "fix/hack" to MiniB3D.mod/Inc/TMesh.bmx at line 1796.
It uses texture flag 512 (which isn't used anyways) as a way to get GL_NEAREST filtering.



And it's not a module yet either, just a file you import.
I'll turn it into a module once I'm happy with it.

Oh, and don't forget that you'll need MiniB3D to use it.