.lit2 format - request for feedback

Discuss programming topics for the various GPL'd game engine sources.
mh
Posts: 2292
Joined: Sat Jan 12, 2008 1:38 am

Re: .lit2 format - request for feedback

Post by mh »

Most of this is @ericw

I suggested including texturemins as well as extents, which may have gotten lost in the noise. That would bring a struct to an equally odd 17 bytes (assuming short). It's hard to see where one byte could be saved (or what else could be used to usefully pad the struct) since everything else is either trivially calculated, dependent on run-time stuff, or not relevant to lighting. The only thing I can reasonably think of is a 3-byte compressed normal, but that can also be taken from surf->plane (and flipped for planeback), and is something that most people won't even use.

Either way, it's looking more like "accept the implementation annoyance on technical grounds" is going to be the lesser evil.

Anyway, the reason for also including texturemins is that it's value is dependent on the same "dot (v, s->texinfo->vecs[j]) + s->texinfo->vecs[j][3]" as produces weird results with extents. I'm not sure if this is even a problem with texturemins, or if it's only extents that need to be corrected. That aside, including texturemins would allow an implementation to skip over CalcSurfaceExtents entirely, which may give a slightly faster map load (something I think is worth trading a little extra disk space for). My research engine also calculates surface bounds in CalcSurfaceExtents, but I can push that to polygon building time just as easy.

My other comments about extents in my previous post were an implementation fault on my part, so you can ignore them.

Bug: offsets needs to be int, not unsigned int, because -1 is a valid offset (indicating that the surf doesn't have any lightmaps).

I've been following the func_msgboard discussion, but since most of it is mapping oriented I haven't really participated.
We had the power, we had the space, we had a sense of time and place
We knew the words, we knew the score, we knew what we were fighting for
Spike
Posts: 2914
Joined: Fri Nov 05, 2004 3:12 am
Location: UK
Contact:

Re: .lit2 format - request for feedback

Post by Spike »

one stance is that if you're byteswapping anyway, then alignment doesn't matter.
the other stance is that 0-intialised padding can be used for later extensions.
or we could store the 2 lower bits of the mins+extents in there which should allow to correct any precision issues with calcsurfaceextents (does float mins make sense? is that ever going to be a problem?).

-1 is a special case. we can call it ~0 instead if you want. either way, you should probably be checking for out-of-range values anyway, instead of just -1.
I really hope the distinction between signed and unsigned offsets never matters... (and can only ever matter in a 64bit build).

brainfart:

Code: Select all

typedef struct litsurf_s
{
    uint offset;        //any invalid value (eg: 0xffffffff) specifies no lighting provided
    byte styles[4];
    byte shift;
    byte extcorrect;  //mins0&96,mins1&24,extent0&5,extent1&3. overwrites calculated bits (careful with the invalid offset)
    signed char overbrightbits;         //0=halflife-alike,1=quake-compat,2=quake3-alike. -1 dark but with higher precision, -2 really dark but with even more precision
    byte extendedflags;
} litsurf_t;
it might be interesting to provide some sort of RLE coding or something, I dunno. it would mess with multiple styles though.
you can argue that overbrightbits should be global, but that would remove the advantage of negative values, and the engine probably has some fixed value anyway (global cvar or something, rather than having to update the scale per-surface). either way, the actual value the hardware sees needs to be somewhat global so that you don't get obviously-weird results with colour clamping (plus you might want to let dlights overbright more than world lighting).

I was thinking about storing formats in that extendedflags byte (ie: 16bit data, or lum-only), but per-surface formats would need to turn offsets into byte offsets rather than consistent with existing .lits (which shouldn't really be a problem) as well as ruin any possibility of using the same offset for the deluxemaps section too (unless those are ALSO 16bit, but that seems needlessly wasteful, and lum-only deluxemaps isn't an option).

hqx almost assumes binary inputs (matching vs non-matching), the input is pixels, the output is curved pixels. or in other words, the stair effect is probably going to look really ugly. give it a go if you dare.
Baker
Posts: 3666
Joined: Tue Mar 14, 2006 5:15 am

Re: .lit2 format - request for feedback

Post by Baker »

Spike wrote:hqx almost assumes binary inputs (matching vs non-matching), the input is pixels, the output is curved pixels. or in other words, the stair effect is probably going to look really ugly. give it a go if you dare.
Result: blocky as hell with stair stepping. :lol:

There is no magic in this world. :D

Tried hqx and this one too http://scale2x.sourceforge.net/index.html both as 2x and 4x.

I guess no one has invented something that can smooth out diagonal lines yet, however part of this might be attributable to how incredibly crummy the source data is.
The night is young. How else can I annoy the world before sunsrise? 8) Inquisitive minds want to know ! And if they don't -- well like that ever has stopped me before ..
mh
Posts: 2292
Joined: Sat Jan 12, 2008 1:38 am

Re: .lit2 format - request for feedback

Post by mh »

A hand-coded bicubic filter in a fragment shader works well for smoothing out stair-steps, it's probably the best you're going to get outside of just using higher resolution source data.

It's slower than regular texture lookups.

Because multiple lightmaps are packed into a single texture they tend to bleed into each other at the edges of surfaces.

And it suffers from the same fundamental problem as any kind of upsampling. With digital data, once something is lost it's gone for ever. So while you can smooth out blockiness, you're not going to be able to restore fine detail. Stuff like this:

Image
We had the power, we had the space, we had a sense of time and place
We knew the words, we knew the score, we knew what we were fighting for
mankrip
Posts: 924
Joined: Fri Jul 04, 2008 3:02 am

Re: .lit2 format - request for feedback

Post by mankrip »

I downloaded the current source and binaries and will try to implement the lightmapped turbulent surfaces. I just hope my setup consisting of an old version of Code:: Blocks + MSVC Toolkit 2003 & MASM will be able to compile it.
Ph'nglui mglw'nafh mankrip Hell's end wgah'nagl fhtagn.
==-=-=-=-=-=-=-=-=-=-==
Dev blog / Twitter / YouTube
mankrip
Posts: 924
Joined: Fri Jul 04, 2008 3:02 am

Re: .lit2 format - request for feedback

Post by mankrip »

ericw wrote:@mankrip, cool - I heard hmap2 and darkplaces support lit water as well.
hmap2 doesn't support it. I did even read its entire source code and confirmed that the TEX_SPECIAL flag in it is still hardcoded to work exactly as in the vanilla q1tools.

There is absolutely no BSP and LIGHT tools that creates lightmapped turbulent surfaces. I did a good search for one a few months ago.

This could be implemented in a few minutes, but I haven't managed to compile the tyrutils code. I've tried cygwin and MinGW, and both gives a makefile error saying "OS not detected". I don't know which tools I need to compile it, and with no Internet access at home I'm a bit helpless.

[edit] I tried downloading cmake, and it fails at 9.71MB. Maybe another low RAM error in my tablet…

And I don't know what else is needed.
Ph'nglui mglw'nafh mankrip Hell's end wgah'nagl fhtagn.
==-=-=-=-=-=-=-=-=-=-==
Dev blog / Twitter / YouTube
ericw
Posts: 92
Joined: Sat Jan 18, 2014 2:11 am

Re: .lit2 format - request for feedback

Post by ericw »

re building tyrutils: sorry you're having trouble with it. I've only used the original Makefile on OS X and Linux. The makefile is a bit complex and I wanted to learn cmake, so I added a new cmake build system.

If you get ahold of cmake, you can have it generate a mingw-based Code::Blocks project like this: (from cmd.exe):

Code: Select all

mkdir cmakebuild
cd cmakebuild
cmake -G "CodeBlocks - MinGW Makefiles" ..
It is also possible to generate a MSVC project, but the code doesn't compile with MSVC currently due to using some gcc extensions.
mankrip
Posts: 924
Joined: Fri Jul 04, 2008 3:02 am

Re: .lit2 format - request for feedback

Post by mankrip »

:) Thanks, I'll try it.
Ph'nglui mglw'nafh mankrip Hell's end wgah'nagl fhtagn.
==-=-=-=-=-=-=-=-=-=-==
Dev blog / Twitter / YouTube
mankrip
Posts: 924
Joined: Fri Jul 04, 2008 3:02 am

Re: .lit2 format - request for feedback

Post by mankrip »

I was looking through the tyrutils source code, and then "Wait, what's that?".

:oops: Its qbsp's -splitspecial option does what I need. It's overkill since it will also split sky surfaces, but that's not a big problem; what matters is that it does the job.

:oops: I'm sorry for kinda hijacking this thread with this. And I didn't have anything to contribute to it, but I trust you guys will make the right decisions.

When the .lit2 specs are finalized I'll try to implement it.
Ph'nglui mglw'nafh mankrip Hell's end wgah'nagl fhtagn.
==-=-=-=-=-=-=-=-=-=-==
Dev blog / Twitter / YouTube
Post Reply