Page 1 of 1

Mysterious Black Pixels

Posted: Sun Aug 23, 2015 3:50 am
by Ghost_Fang
I'm not sure if this belongs here in Engine Programming or not. I think it does because the issue seems to be only apparent in my DQuake PSP engine. Other tools such as Crafty BSP viewer has no such anomalies. It appears to me that those might be "fullbright" colors not rendering right (to which i have no use for). This is a HLBSP map made in Hammer 3.5.3 and i don't use wadinclude. Has anyone ever experienced this and have fixed/remedied this?

In Game:
Image

Source Texture:
Image

Re: Mysterious Black Pixels

Posted: Sun Aug 23, 2015 5:24 am
by Spike
HLBSP has per-texture palettes (meaning the last 32 indexes could be any colour). it also has no fullbrights.
it also has toolchain licensing issues.

Re: Mysterious Black Pixels

Posted: Sun Aug 23, 2015 5:31 am
by Ghost_Fang
I figured as much, it was just a speculation i had because I first noticed it on a light texture that I have. I scrubbed through the engine to try to find ANYTHING about pixel manipulation i cannot find any.

Re: Mysterious Black Pixels

Posted: Sun Aug 23, 2015 5:33 am
by Ghost_Fang
(cannot find the edit button if there is one anymore)
So what would be my issue then? Have you seen anything like this?

Re: Mysterious Black Pixels

Posted: Sun Aug 23, 2015 7:04 am
by Spike
if its nulling out the last 32 palette indexes to black because they're within the last vid.fullbright pixels, and then not generating a fullbright image because its an hlbsp, then that might explain it quite well. its certainly possible.
engines are vastly different nowadays so I can't give you many more hints other than vid.fullbright or the number 32 if its hardcoded, or possibly 224. also check to see if it has some cvar to disable fullbrights (not to be confused with the fully-lit lightmaps, of course). good luck...

Re: Mysterious Black Pixels

Posted: Sun Aug 23, 2015 1:24 pm
by frag.machine
You can open the texture in any image editor and check the pixel colors to confirm (or discard) the palette theory. But if the dot patterns are constant in all places where the texture is used I can't think of any other suspect.

Re: Mysterious Black Pixels

Posted: Mon Aug 24, 2015 2:43 am
by ceriux
perhaps it has to do with the texture compression used on psp engines?

Re: Mysterious Black Pixels

Posted: Tue Aug 25, 2015 3:53 pm
by drm_wayne
Kurok was the first engine who fucked this up for having transparency on the last color on map textures,
it seems also affect the HLBSP loader.
Its not an compression artefact, since DQuake only compresses truecolor images to DXT5.

I looked at the DQuake src and i found this in VID_SetPalette (video_hardware.cpp):

Code: Select all

	// Color 255 is transparent black.
	// This is a bit of a dirty hack.
	d_8to24table[255] = 0;
Maybe modify it for HLBSP palettes?

Re: Mysterious Black Pixels

Posted: Tue Aug 25, 2015 9:03 pm
by frag.machine
Actually index 255 in Quake palette was always meant to be a transparent, null color (all 2D graphics such fonts, menu elements and even sprites use it this way). Kurok only extended the concept to alias models. The problem Ghost_Fang reported seems related to fullbrights, as Spike observed.

Re: Mysterious Black Pixels

Posted: Wed Aug 26, 2015 12:20 am
by Spike
conchars(read: quake's font) uses 0 for transparent. because special cases are fun.
and 255 is pink, not black.
255 is only 'black' if you consider it transparent, in which case you should be using premultiplied alpha which results in pink*0=0=black. obviously this doesn't apply if its an 8-to-24 lookup table as that has no alpha channel and thus cannot be transparent and thus must be pink, not black.
not supporting premultiplied alpha will result in halos. yes, black halos are generally better than pink halos, but any kind of halo is still a bug - even the filling described below would be preferable to black halos.

For bsp textures, palette index 255 is transparent ONLY if the texture's name has a leading {. Due to depth issues with alpha blending, premultiplied alpha is no longer reliable, and thus engines will probably switch to alpha testing, and in order to avoid obvious halos, these transparent pixels should be filled in with the average of their neighbouring (valid) pixels (but with alpha 0).

Note that halflife has PER-TEXTURE palettes, thus some bug inside your 'VID_SetPalette' function won't affect your halflife textures. Because of the filling described above, it shouldn't affect your q1 textures with { in them either, but it WILL affect valid q1bsp textures/skins that happen to use index 255.

Obviously, because halflife textures have their own per-texture palette, and halflife never supported fullbrights, any attempt to utilise fullbrights in said halflife textures is a bug.

Obviously there could be all sorts of bugs in random projects, but these are the most likely ones that I can think of.

Re: Mysterious Black Pixels

Posted: Wed Aug 26, 2015 8:27 am
by drm_wayne
It has nothing to do with fullbright, i already found the issue.
Its in the textureloader function called from HLBSP (GL_LoadPalTex),
caused by "texture.palette[255] = 0".
This should be only used for "{" textures, and not for regular textureimages.
The engine author was really lazy and sloppy (when loading maps hes calling
Mod_LoadTextures 2 TIMES!!!)

Code: Select all

/*
================
GL_LoadPalTex
================
*/
int GL_LoadPalTex (const char *identifier, int width, int height, const byte *data, qboolean stretch_to_power_of_two, int filter, int mipmap_level, byte *palette, int paltype)
{
	int texture_index = -1;

	tex_scale_down = r_tex_scale_down.value == qtrue;
	// See if the texture is already present.
	if (identifier[0])
	{
		for (int i = 0; i < MAX_GLTEXTURES; ++i)
		{
			if (gltextures_used[i] == true)
			{
				const gltexture_t& texture = gltextures[i];
				if (!strcmp (identifier, texture.identifier))
				{
					return i;
				}
			}
		}
	}

	// Out of textures?
	if (numgltextures == MAX_GLTEXTURES)
	{
		Sys_Error("Out of OpenGL textures");
	}

	// Use the next available texture.
	numgltextures++;
	texture_index = numgltextures;

	for (int i = 0; i < MAX_GLTEXTURES; ++i)
	{
		if (gltextures_used[i] == false)
		{
			texture_index = i;
			break;
		}
	}
	gltexture_t& texture = gltextures[texture_index];
	gltextures_used[texture_index] = true;

	// Fill in the source data.
	strcpy(texture.identifier, identifier);
	texture.original_width			= width;
	texture.original_height			= height;
	texture.stretch_to_power_of_two	= stretch_to_power_of_two != qfalse;

	// Fill in the texture description.
	texture.format			= GU_PSM_T8;
	texture.filter			= filter;
	texture.mipmaps			= mipmap_level;
    texture.swizzle         = GU_TRUE;
	texture.bpp             = 1;

	// Upload the Palette
    if((paltype == PAL_RGB  && palette) ||
       (paltype == PAL_RGBA && palette) ||
	   (paltype == PAL_Q2   && palette == NULL) ||
	   (paltype == PAL_H2   && palette == NULL) )
    {
#ifndef STATIC_PAL
        if(paltype == PAL_Q2)
	    {
              texture.palette = d_8to24tableQ2; //hard coded palette
		}
	    else if(paltype == PAL_H2)
	    {
              texture.palette = d_8to24tableH2; //hard coded palette
		}
		else
		{
			texture.palette = static_cast<ScePspRGBA8888*>(memalign(16, sizeof(ScePspRGBA8888) * 256));
			if(!texture.palette)
			{
	            Sys_Error("Out of RAM for palettes.");
			}
#endif
			if(paltype == PAL_RGBA)
			{
				  // Convert the palette to PSP format.
			      for (ScePspRGBA8888* color = &texture.palette[0]; color < &texture.palette[256]; ++color)
				  {
					const unsigned int r = gammatable[*palette++];
					const unsigned int g = gammatable[*palette++];
					const unsigned int b = gammatable[*palette++];
					const unsigned int a = gammatable[*palette++];
					*color = GU_RGBA(r, g, b, a);
				  }
	        }
			else if(paltype == PAL_RGB)
			{
				  // Convert the palette to PSP format.
			      for (ScePspRGBA8888* color = &texture.palette[0]; color < &texture.palette[256]; ++color)
				  {	  
					const unsigned int r = gammatable[*palette++];
					const unsigned int g = gammatable[*palette++];
					const unsigned int b = gammatable[*palette++];
					*color = GU_RGBA(r, g, b, 0xff);
				  }
		    }
#ifndef STATIC_PAL
		}
#endif
		texture.palette[255] = 0;  //alpha color
		texture.palette_active  = qtrue;
	}
	else
    {
	    Sys_Error("GL_LoadPalTex: Unknow palette type");
	}


	if (tex_scale_down == true && texture.stretch_to_power_of_two == true)
	{
		texture.width			= std::max(round_down(width), 32U);
		texture.height			= std::max(round_down(height),32U);
	}
	else
	{
		texture.width			= std::max(round_up(width), 32U);
		texture.height			= std::max(round_up(height),32U);
	}

	for (int i=0; i <= mipmap_level;i++)
	{
		int div = (int) powf(2,i);
		if ((texture.width / div) > 16 && (texture.height / div) > 16 )
		{
			texture.mipmaps = i;
		}
	}

	// Do we really need to resize the texture?
	if (texture.stretch_to_power_of_two)
	{
		// Not if the size hasn't changed.
		texture.stretch_to_power_of_two = (texture.width != width) || (texture.height != height);
	}

	Con_DPrintf("Loading TEX_PAL: %s [%dx%d](%0.2f KB)\n",texture.identifier,texture.width,texture.height, (float) ((texture.width*texture.height)/1024) + 256);

	// Allocate the RAM.
	std::size_t buffer_size = texture.width * texture.height;

	if (texture.mipmaps > 0)
	{
		int size_incr = buffer_size/4;
		for (int i= 1;i <= texture.mipmaps;i++)
		{
			buffer_size += size_incr;
			size_incr = size_incr/4;
		}
	}

	texture.ram	= static_cast<texel*>(memalign(16, buffer_size));

	if (!texture.ram)
	{
		Sys_Error("Out of RAM for textures.");
	}

	// Allocate the VRAM.
	texture.vram = static_cast<texel*>(valloc(buffer_size));

	// Upload the texture.
	GL_Upload8(texture_index, data, width, height);

	if (texture.vram && texture.ram)
	{
		free(texture.ram);
		texture.ram = NULL;
	}
	// Done.
	return texture_index;
}

Re: Mysterious Black Pixels

Posted: Mon Mar 07, 2016 3:43 pm
by Ghost_Fang
Sorry for taking a long time. Been out of the quake scene for a while. Life took over.

i have confirmed drm_wayne's answer as the real culprit.

setting

Code: Select all

texture.palette[255] = texture.palette[255];
does in fact get rid of the black pixels. However, what would i have to do to determine if that index SHOULD be 0? Is there a way i could access the RGB of that index to determine if the color is 0, 0, 255? That way i can still have my "{" transparent blue textures?

thanks btw drm_wayne. I have scoured the code trying to find the cause of the issue


EDIT:
I fixed it myself. Thank you drm_wayne for pointing me in the right direction. This is how i fixed it:

Code: Select all

if (!Q_strncmp(texture.identifier, "{",1))
	texture.palette[255] = 0; //alpha color
else
	texture.palette[255] = texture.palette[255];
The black dots are gone and the blue alpha works again. So dquake users there is your fix if you were having the same issue. I didnt see any artifacts or issues, but please let me know if the way i did it will cause any issues down the road.

Re: Mysterious Black Pixels

Posted: Tue Mar 08, 2016 8:54 am
by drm_wayne
glad i could help ;)

you can also try to upload all 8bpp paltextures to the PSPGU with the built-in DXT compressor,
this will save alot of ram.

Re: Mysterious Black Pixels

Posted: Tue Mar 08, 2016 10:54 pm
by Ghost_Fang
drm_wayne wrote: you can also try to upload all 8bpp paltextures to the PSPGU with the built-in DXT compressor,
this will save alot of ram.
I could try to look into it, but im sure thats out of my league (currently). I knew enough basic C to get by and hardly anything about PSPGU

Re: Mysterious Black Pixels

Posted: Sat Mar 19, 2016 2:11 am
by jitspoe
#BlackPixelsMatter

Sorry. Couldn't resist.