Mysterious Black Pixels

Discuss programming topics for the various GPL'd game engine sources.
Post Reply
Ghost_Fang
Posts: 336
Joined: Thu Nov 12, 2009 4:37 am

Mysterious Black Pixels

Post by Ghost_Fang »

I'm not sure if this belongs here in Engine Programming or not. I think it does because the issue seems to be only apparent in my DQuake PSP engine. Other tools such as Crafty BSP viewer has no such anomalies. It appears to me that those might be "fullbright" colors not rendering right (to which i have no use for). This is a HLBSP map made in Hammer 3.5.3 and i don't use wadinclude. Has anyone ever experienced this and have fixed/remedied this?

In Game:
Image

Source Texture:
Image
Spike
Posts: 2914
Joined: Fri Nov 05, 2004 3:12 am
Location: UK
Contact:

Re: Mysterious Black Pixels

Post by Spike »

HLBSP has per-texture palettes (meaning the last 32 indexes could be any colour). it also has no fullbrights.
it also has toolchain licensing issues.
Ghost_Fang
Posts: 336
Joined: Thu Nov 12, 2009 4:37 am

Re: Mysterious Black Pixels

Post by Ghost_Fang »

I figured as much, it was just a speculation i had because I first noticed it on a light texture that I have. I scrubbed through the engine to try to find ANYTHING about pixel manipulation i cannot find any.
Ghost_Fang
Posts: 336
Joined: Thu Nov 12, 2009 4:37 am

Re: Mysterious Black Pixels

Post by Ghost_Fang »

(cannot find the edit button if there is one anymore)
So what would be my issue then? Have you seen anything like this?
Spike
Posts: 2914
Joined: Fri Nov 05, 2004 3:12 am
Location: UK
Contact:

Re: Mysterious Black Pixels

Post by Spike »

if its nulling out the last 32 palette indexes to black because they're within the last vid.fullbright pixels, and then not generating a fullbright image because its an hlbsp, then that might explain it quite well. its certainly possible.
engines are vastly different nowadays so I can't give you many more hints other than vid.fullbright or the number 32 if its hardcoded, or possibly 224. also check to see if it has some cvar to disable fullbrights (not to be confused with the fully-lit lightmaps, of course). good luck...
frag.machine
Posts: 2126
Joined: Sat Nov 25, 2006 1:49 pm

Re: Mysterious Black Pixels

Post by frag.machine »

You can open the texture in any image editor and check the pixel colors to confirm (or discard) the palette theory. But if the dot patterns are constant in all places where the texture is used I can't think of any other suspect.
I know FrikaC made a cgi-bin version of the quakec interpreter once and wrote part of his website in QuakeC :) (LordHavoc)
ceriux
Posts: 2230
Joined: Sat Sep 06, 2008 3:30 pm
Location: Indiana, USA

Re: Mysterious Black Pixels

Post by ceriux »

perhaps it has to do with the texture compression used on psp engines?
drm_wayne
Posts: 232
Joined: Sat Feb 11, 2012 5:47 pm

Re: Mysterious Black Pixels

Post by drm_wayne »

Kurok was the first engine who fucked this up for having transparency on the last color on map textures,
it seems also affect the HLBSP loader.
Its not an compression artefact, since DQuake only compresses truecolor images to DXT5.

I looked at the DQuake src and i found this in VID_SetPalette (video_hardware.cpp):

Code: Select all

	// Color 255 is transparent black.
	// This is a bit of a dirty hack.
	d_8to24table[255] = 0;
Maybe modify it for HLBSP palettes?
frag.machine
Posts: 2126
Joined: Sat Nov 25, 2006 1:49 pm

Re: Mysterious Black Pixels

Post by frag.machine »

Actually index 255 in Quake palette was always meant to be a transparent, null color (all 2D graphics such fonts, menu elements and even sprites use it this way). Kurok only extended the concept to alias models. The problem Ghost_Fang reported seems related to fullbrights, as Spike observed.
I know FrikaC made a cgi-bin version of the quakec interpreter once and wrote part of his website in QuakeC :) (LordHavoc)
Spike
Posts: 2914
Joined: Fri Nov 05, 2004 3:12 am
Location: UK
Contact:

Re: Mysterious Black Pixels

Post by Spike »

conchars(read: quake's font) uses 0 for transparent. because special cases are fun.
and 255 is pink, not black.
255 is only 'black' if you consider it transparent, in which case you should be using premultiplied alpha which results in pink*0=0=black. obviously this doesn't apply if its an 8-to-24 lookup table as that has no alpha channel and thus cannot be transparent and thus must be pink, not black.
not supporting premultiplied alpha will result in halos. yes, black halos are generally better than pink halos, but any kind of halo is still a bug - even the filling described below would be preferable to black halos.

For bsp textures, palette index 255 is transparent ONLY if the texture's name has a leading {. Due to depth issues with alpha blending, premultiplied alpha is no longer reliable, and thus engines will probably switch to alpha testing, and in order to avoid obvious halos, these transparent pixels should be filled in with the average of their neighbouring (valid) pixels (but with alpha 0).

Note that halflife has PER-TEXTURE palettes, thus some bug inside your 'VID_SetPalette' function won't affect your halflife textures. Because of the filling described above, it shouldn't affect your q1 textures with { in them either, but it WILL affect valid q1bsp textures/skins that happen to use index 255.

Obviously, because halflife textures have their own per-texture palette, and halflife never supported fullbrights, any attempt to utilise fullbrights in said halflife textures is a bug.

Obviously there could be all sorts of bugs in random projects, but these are the most likely ones that I can think of.
drm_wayne
Posts: 232
Joined: Sat Feb 11, 2012 5:47 pm

Re: Mysterious Black Pixels

Post by drm_wayne »

It has nothing to do with fullbright, i already found the issue.
Its in the textureloader function called from HLBSP (GL_LoadPalTex),
caused by "texture.palette[255] = 0".
This should be only used for "{" textures, and not for regular textureimages.
The engine author was really lazy and sloppy (when loading maps hes calling
Mod_LoadTextures 2 TIMES!!!)

Code: Select all

/*
================
GL_LoadPalTex
================
*/
int GL_LoadPalTex (const char *identifier, int width, int height, const byte *data, qboolean stretch_to_power_of_two, int filter, int mipmap_level, byte *palette, int paltype)
{
	int texture_index = -1;

	tex_scale_down = r_tex_scale_down.value == qtrue;
	// See if the texture is already present.
	if (identifier[0])
	{
		for (int i = 0; i < MAX_GLTEXTURES; ++i)
		{
			if (gltextures_used[i] == true)
			{
				const gltexture_t& texture = gltextures[i];
				if (!strcmp (identifier, texture.identifier))
				{
					return i;
				}
			}
		}
	}

	// Out of textures?
	if (numgltextures == MAX_GLTEXTURES)
	{
		Sys_Error("Out of OpenGL textures");
	}

	// Use the next available texture.
	numgltextures++;
	texture_index = numgltextures;

	for (int i = 0; i < MAX_GLTEXTURES; ++i)
	{
		if (gltextures_used[i] == false)
		{
			texture_index = i;
			break;
		}
	}
	gltexture_t& texture = gltextures[texture_index];
	gltextures_used[texture_index] = true;

	// Fill in the source data.
	strcpy(texture.identifier, identifier);
	texture.original_width			= width;
	texture.original_height			= height;
	texture.stretch_to_power_of_two	= stretch_to_power_of_two != qfalse;

	// Fill in the texture description.
	texture.format			= GU_PSM_T8;
	texture.filter			= filter;
	texture.mipmaps			= mipmap_level;
    texture.swizzle         = GU_TRUE;
	texture.bpp             = 1;

	// Upload the Palette
    if((paltype == PAL_RGB  && palette) ||
       (paltype == PAL_RGBA && palette) ||
	   (paltype == PAL_Q2   && palette == NULL) ||
	   (paltype == PAL_H2   && palette == NULL) )
    {
#ifndef STATIC_PAL
        if(paltype == PAL_Q2)
	    {
              texture.palette = d_8to24tableQ2; //hard coded palette
		}
	    else if(paltype == PAL_H2)
	    {
              texture.palette = d_8to24tableH2; //hard coded palette
		}
		else
		{
			texture.palette = static_cast<ScePspRGBA8888*>(memalign(16, sizeof(ScePspRGBA8888) * 256));
			if(!texture.palette)
			{
	            Sys_Error("Out of RAM for palettes.");
			}
#endif
			if(paltype == PAL_RGBA)
			{
				  // Convert the palette to PSP format.
			      for (ScePspRGBA8888* color = &texture.palette[0]; color < &texture.palette[256]; ++color)
				  {
					const unsigned int r = gammatable[*palette++];
					const unsigned int g = gammatable[*palette++];
					const unsigned int b = gammatable[*palette++];
					const unsigned int a = gammatable[*palette++];
					*color = GU_RGBA(r, g, b, a);
				  }
	        }
			else if(paltype == PAL_RGB)
			{
				  // Convert the palette to PSP format.
			      for (ScePspRGBA8888* color = &texture.palette[0]; color < &texture.palette[256]; ++color)
				  {	  
					const unsigned int r = gammatable[*palette++];
					const unsigned int g = gammatable[*palette++];
					const unsigned int b = gammatable[*palette++];
					*color = GU_RGBA(r, g, b, 0xff);
				  }
		    }
#ifndef STATIC_PAL
		}
#endif
		texture.palette[255] = 0;  //alpha color
		texture.palette_active  = qtrue;
	}
	else
    {
	    Sys_Error("GL_LoadPalTex: Unknow palette type");
	}


	if (tex_scale_down == true && texture.stretch_to_power_of_two == true)
	{
		texture.width			= std::max(round_down(width), 32U);
		texture.height			= std::max(round_down(height),32U);
	}
	else
	{
		texture.width			= std::max(round_up(width), 32U);
		texture.height			= std::max(round_up(height),32U);
	}

	for (int i=0; i <= mipmap_level;i++)
	{
		int div = (int) powf(2,i);
		if ((texture.width / div) > 16 && (texture.height / div) > 16 )
		{
			texture.mipmaps = i;
		}
	}

	// Do we really need to resize the texture?
	if (texture.stretch_to_power_of_two)
	{
		// Not if the size hasn't changed.
		texture.stretch_to_power_of_two = (texture.width != width) || (texture.height != height);
	}

	Con_DPrintf("Loading TEX_PAL: %s [%dx%d](%0.2f KB)\n",texture.identifier,texture.width,texture.height, (float) ((texture.width*texture.height)/1024) + 256);

	// Allocate the RAM.
	std::size_t buffer_size = texture.width * texture.height;

	if (texture.mipmaps > 0)
	{
		int size_incr = buffer_size/4;
		for (int i= 1;i <= texture.mipmaps;i++)
		{
			buffer_size += size_incr;
			size_incr = size_incr/4;
		}
	}

	texture.ram	= static_cast<texel*>(memalign(16, buffer_size));

	if (!texture.ram)
	{
		Sys_Error("Out of RAM for textures.");
	}

	// Allocate the VRAM.
	texture.vram = static_cast<texel*>(valloc(buffer_size));

	// Upload the texture.
	GL_Upload8(texture_index, data, width, height);

	if (texture.vram && texture.ram)
	{
		free(texture.ram);
		texture.ram = NULL;
	}
	// Done.
	return texture_index;
}
Ghost_Fang
Posts: 336
Joined: Thu Nov 12, 2009 4:37 am

Re: Mysterious Black Pixels

Post by Ghost_Fang »

Sorry for taking a long time. Been out of the quake scene for a while. Life took over.

i have confirmed drm_wayne's answer as the real culprit.

setting

Code: Select all

texture.palette[255] = texture.palette[255];
does in fact get rid of the black pixels. However, what would i have to do to determine if that index SHOULD be 0? Is there a way i could access the RGB of that index to determine if the color is 0, 0, 255? That way i can still have my "{" transparent blue textures?

thanks btw drm_wayne. I have scoured the code trying to find the cause of the issue


EDIT:
I fixed it myself. Thank you drm_wayne for pointing me in the right direction. This is how i fixed it:

Code: Select all

if (!Q_strncmp(texture.identifier, "{",1))
	texture.palette[255] = 0; //alpha color
else
	texture.palette[255] = texture.palette[255];
The black dots are gone and the blue alpha works again. So dquake users there is your fix if you were having the same issue. I didnt see any artifacts or issues, but please let me know if the way i did it will cause any issues down the road.
drm_wayne
Posts: 232
Joined: Sat Feb 11, 2012 5:47 pm

Re: Mysterious Black Pixels

Post by drm_wayne »

glad i could help ;)

you can also try to upload all 8bpp paltextures to the PSPGU with the built-in DXT compressor,
this will save alot of ram.
Ghost_Fang
Posts: 336
Joined: Thu Nov 12, 2009 4:37 am

Re: Mysterious Black Pixels

Post by Ghost_Fang »

drm_wayne wrote: you can also try to upload all 8bpp paltextures to the PSPGU with the built-in DXT compressor,
this will save alot of ram.
I could try to look into it, but im sure thats out of my league (currently). I knew enough basic C to get by and hardly anything about PSPGU
jitspoe
Posts: 217
Joined: Mon Jan 17, 2005 5:27 am

Re: Mysterious Black Pixels

Post by jitspoe »

#BlackPixelsMatter

Sorry. Couldn't resist.
Post Reply