QBSP-VIS-RAD compiling process detailed documentation?

Discuss programming topics for the various GPL'd game engine sources.
Post Reply
rec
Posts: 4
Joined: Mon Nov 10, 2014 2:56 pm

QBSP-VIS-RAD compiling process detailed documentation?

Post by rec »

Is there any good documentation about how the BSP is compiled?
Especially LIGHT stage.

I want to know the technique how the sample points created, all down to writing the lightmap to the bsp.

I took the original Quake tools from Id Software, and i managed to run them in vs2017.
The src is small (easier for debugging), but is poorly commented.

QBSP is working ok, but the LIGHT compiler is multi-threaded with Pthreads, and Windows is skipping the main code.

Can you point me to some docs (if any), or to some not so heavily modded tools, which the src is commented?

Tnx, rec
mh
Posts: 2292
Joined: Sat Jan 12, 2008 1:38 am

Re: QBSP-VIS-RAD compiling process detailed documentation?

Post by mh »

Michael Abrash’s Graphics Programming Black Book is a source of information, but don't expect it to take you on a line-by-line trip through the code. It's available for free online here: http://www.drdobbs.com/parallel/graphic ... /184404919

Relevant to lighting are chapters 68 and 69.

The light tool itself doesn't do anything special. It just divides surfaces into 16x16 blocks then does a raytrace from a lightsource to a block, using a standard-ish NdotL formula with linear attenuation to accumulate light.
We had the power, we had the space, we had a sense of time and place
We knew the words, we knew the score, we knew what we were fighting for
rec
Posts: 4
Joined: Mon Nov 10, 2014 2:56 pm

Re: QBSP-VIS-RAD compiling process detailed documentation?

Post by rec »

mh wrote: The light tool itself doesn't do anything special. It just divides surfaces into 16x16 blocks then does a raytrace from a lightsource to a block, using a standard-ish NdotL formula with linear attenuation to accumulate light.
Thanks mh!

I managed to remove pthreads (I was skipping functions with F10 all the time doh!).
There was #ifdef __alpha for pthread, and it never even got to LightThread()
So i changed

Code: Select all

RunThreadsOn(LightThread);
to

Code: Select all

LightThread();

One small problem:
Image

This is just a box map with only one light, dist and total are ok, idk how it scrambled like this lol
ericw
Posts: 92
Joined: Sat Jan 18, 2014 2:11 am

Re: QBSP-VIS-RAD compiling process detailed documentation?

Post by ericw »

I'm not sure if it would cause results like that, but LightThread takes a pointer argument (it's unused), so call it like this:

Code: Select all

LightThread(NULL);
I've got a fork of tyrutils, but it's straying quite far from the original utils so it may not be useful to study (and I've made a bit of a mess admittedly). However, it has a win32 implementation of the threading functions in the Quake utils that might be useful if you want to restore threading:
https://github.com/ericwa/tyrutils-eric ... threads.cc

Aside from what mh said, the main clever thing that the light tool does is in CalcPoints: to avoid sample points going inside walls, it traces a ray from a point at the face centroid (nudged off the face) to where the sample point should be; so if there's a wall in the way, this "test" ray hits the wall. In that case the sample point is moved to the hit position and nudged slightly towards the centroid.

Without something like that, you'd have black fringes everywhere.
If you didn't let the sample points fall outside the face polygon, you wouldn't have the problem of sample points going inside walls, but it would mess up smooth transitions between faces. (similar to cutting an image in half, resizing both halves 50% and joining them back together will leave a seam)
rec
Posts: 4
Joined: Mon Nov 10, 2014 2:56 pm

Re: QBSP-VIS-RAD compiling process detailed documentation?

Post by rec »

A-ha!


Image

Eric, thanks for the tip, i almost decided to implement win threading, but recompiled it once more, and it works.
And idk why, I cannot reproduce the bug.
Works both with LightThread(NULL); and LightThread();

Anyway, this is the result from the profiling on E1M1:

Image


Question:
How do I increase the lightmap size by 4?

These are the sample points I presume:

Code: Select all

mins[i] = floor(mins[i]/16);
maxs[i] = ceil(maxs[i]/16);
Changed them to 8 and in

Code: Select all

if (l->texsize[i] > 17)
			Error ("Bad surface extents");
changed 17 to 29, but the lightmap is fkd.

Is this hardcoded in the engine?
Spike
Posts: 2914
Joined: Fri Nov 05, 2004 3:12 am
Location: UK
Contact:

Re: QBSP-VIS-RAD compiling process detailed documentation?

Post by Spike »

the vanilla software renderer had a strict 1:16 ratio of lightmap:texels (for mip level 0, which also means all wall textures must be a multiple of 16). This ratio persists into glquake despite glquake using no surface cache. The file format basically says absolutely nothing about lightmap coords or sizes, they're instead inferred by rounding the regular texture coords and figuring out the extents from that, with floats which are somewhat imprecise and glitches out if you're not really careful (x87 maths is often performed using 80 bits even for 32bit floats, which can be a problem when it gets ported to other cpus/compilers). The light tool and the engine should be using the same maths, and thus should both calculate the same extents for the surface's lightmaps and thus the same lightmap width+height.
texture extents limited to (16+1) in each axis. the +1 allows for interpolation on the side. for some reason the limit was bumped by 1 in glquake, probably to try to hide precision issues. This means that the qbsp MUST subdivide each surface into a maximum of 256*256 texel blocks (larger map textures will just result in two surfaces instead).
Some engines increase the maximum lightmap size to 256*256, which means surfaces can get subdivided to up to 4080 (256*16-16) texels max (which of course crashes other engines). Note that surface subdivision happens in the qbsp util, not the light util. Also note that many surface (read: sky and turb/water) are not lightmapped, and thus do not need to be subdivided by qbsp. Note that even the vanilla qbsp tool had a commandline argument to control the max post-subdivision size.

lit2 has per-surface scales, while bspx has an optional lightmap-scale lump that overrides the 1:16 ratio in engines that recognise it, but support for these is pretty much limited to just fte + tyrutils-ericw, and even then its not well tested and disabled by default.
There's /16 and >>4 in a few different places inside the glquake code.

You can compile the map with -extra and the light util will instead calculate 4 points per luxel instead of 1. It'll then average them so as to not violate the file format. The result is smoother lighting.

Threading the lighting shouldn't be too complicated. About the only thing you'll need mutexes for is allocating output file space and figuring out which surface to light next. In fact, you could probably get it all done with atomic_fetch_and_add without any mutexes. Just avoid using globals.
Post Reply