Revelation Test

Discuss anything not covered by any of the other categories.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Productivity is a state of mind.
nbohr1more
Posts: 54
Joined: Fri Dec 09, 2011 7:04 am

Re: Revelation Test

Post by nbohr1more »

May I request a compile too? :D
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Productivity is a state of mind.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Next version adds nedmalloc support.

nedmalloc is a very fast implementation of malloc and friends, it also sports fast implementations of new and free and can emulate MMAP behaviour on windows for very fast paging.

first test shows loading times get a nice boost :)

nedmalloc can also be used on linux mac and other OS so its crossplatform.

its also usable to debug malloc calls.
Productivity is a state of mind.
qbism
Posts: 1236
Joined: Thu Nov 04, 2004 5:51 am
Contact:

Re: Revelation Test

Post by qbism »

This build looks great with no shadow cut-off issues.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Glad you like it :)

Took me long enough to get it working correctly hehe.

Shadow cut off still happens but i been working towards minimizing it.
Productivity is a state of mind.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

This one was newer used, before now.

Code: Select all

/*
==================
Sys_DestroyThread
==================
*/
void Sys_DestroyThread( xthreadInfo &info ) {
	DWORD	dwWaitResult;
	HANDLE	threadHandle[MAX_THREADS];
	// no threads running so nothing to kill.
	if( !info.threadHandle ) {
		return;
	}
	// get number of threads to wait for.
	for (int count = 0; count < MAX_THREADS; count++ ) {
		// create an array of handles for WaitForMultipleObjects.
		threadHandle[count] = info.threadHandle;
		// wait for the handle to be signaled.
		dwWaitResult = ::WaitForMultipleObjects( count, threadHandle, TRUE, INFINITE );
		// signal handlers for WaitForMultipleObjects.	
		switch( dwWaitResult ) {
			case WAIT_ABANDONED_0:
				idLib::common->Printf( "Mutex object was not released by the thread that owned the mutex object before the owning thread terminates...\n" );
				break;
			case WAIT_OBJECT_0:
				idLib::common->Printf( "The child thread state was signaled!\n" );
				break;
			case WAIT_TIMEOUT:
				idLib::common->Printf( "Time-out interval elapsed, and the child thread's state is nonsignaled.\n" );
				break;
			case WAIT_FAILED:
				idLib::common->FatalError( "WaitForMultipleObjects() failed, error %u\n", GetLastError() );
				ExitProcess( 0 );
		}
		// close the handle and set it back to NULL.
		if ( CloseHandle( threadHandle ) != FALSE ) {
			threadHandle[count] = NULL;
		}
	}
}
Used to wait for the thread handles to signal if they are done working.
BFG's is better though :) but this will do untill i port that.

used here ->

Code: Select all

/*
=================
idFileSystemLocal::StartBackgroundReadThread
=================
*/
void idFileSystemLocal::StartBackgroundDownloadThread() {
	if( !backgroundThread.threadHandle ) {
		Sys_CreateThread( ( xthread_t )BackgroundDownloadThread, NULL, THREAD_NORMAL, backgroundThread, "backgroundDownload", g_threads, &g_thread_count );
		if( !backgroundThread.threadHandle ) {
			common->Warning( "idFileSystemLocal::StartBackgroundDownloadThread: failed" );
		}
	} else {
		common->Printf( "background thread already running\n" );
		Sys_DestroyThread( backgroundThread );
	}
}
and here ->

Code: Select all

/*
==============
Sys_Quit
==============
*/
void Sys_Quit( void ) {
	timeEndPeriod( 1 );
	Sys_ShutdownInput();
	Sys_DestroyConsole();
	Sys_DestroyThread( threadInfo );
}
also here ->

Code: Select all

/*
==============
Sys_StartAsyncThread

Start the thread that will call idCommon::Async()
==============
*/
void Sys_StartAsyncThread( void ) {
	// create an auto-reset event that happens 60 times a second
	hTimer = CreateWaitableTimer( NULL, false, NULL );
	if( !hTimer ) {
		common->Error( "idPacketServer::Spawn: CreateWaitableTimer failed" );
	}
	LARGE_INTEGER	t;
	SYSTEM_INFO		info;
	// check number of processors
	GetSystemInfo( &info );
	t.HighPart = t.LowPart = 0;
	SetWaitableTimer( hTimer, &t, USERCMD_MSEC, NULL, NULL, TRUE );
	if ( !threadInfo.threadHandle ) {
		Sys_CreateThread( ( xthread_t )Sys_AsyncThread, NULL, THREAD_ABOVE_NORMAL, threadInfo, "Async", g_threads,  &g_thread_count );
	} else {
		// thread allready running.
		Sys_DestroyThread( threadInfo );
	}
#ifdef SET_THREAD_AFFINITY
	switch( info.dwNumberOfProcessors ) {
		case 2:
			SetThreadAffinityMask( threadInfo.threadHandle, ( 2 << info.dwNumberOfProcessors ) );
			break;
		case 4:
			SetThreadAffinityMask( threadInfo.threadHandle, ( 4 << info.dwNumberOfProcessors ) );
			break;
		default:
			break;
	}
#endif
	if( !threadInfo.threadHandle ) {
		common->Error( "Sys_StartAsyncThread: failed" );
	}
}
Productivity is a state of mind.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Old thread routine probably wont get any better than what i currently have :) i increased the number of threads to 16 so people with a 6 core HT enabled corei7 can really feel the difference now.
ofc it still works with less than that :) but its a nice option to have.

i also tried out my idea with checking shader flags to fix ssao, unfortunatly it does not work :S because the image routine used to hack out the depth image is wrong for this kinda work but i got a pointer on how to do it by qeffects.

Code: Select all

			qglBindTexture( GL_TEXTURE_RECTANGLE_ARB, m_uiScreenDepth );
			qglTexParameteri( GL_TEXTURE_RECTANGLE_ARB, GL_DEPTH_TEXTURE_MODE_ARB, GL_LUMINANCE );
			qglTexParameteri( GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_WRAP_S, ext.edgeClampMode );
			qglTexParameteri( GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_WRAP_T, ext.edgeClampMode );
			qglTexParameteri( GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_MAG_FILTER, GL_NEAREST );
			qglTexParameteri( GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_MIN_FILTER, GL_NEAREST );
			qglTexImage2D( GL_TEXTURE_RECTANGLE_ARB, 0, GL_DEPTH_COMPONENT24, m_3DViewport[2], m_3DViewport[3], 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_BYTE, 0 );
Still need to make a new function to use with it.
Productivity is a state of mind.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Ok first test looks promissing :) i can now copy the depth buffer.
Still some trouble though as the depthbuffer image bleeds into other postfx effects making things extremely bright.
Guess ill have to use a seperate pointer than "_currentRender" to avoid this.
Productivity is a state of mind.
nbohr1more
Posts: 54
Joined: Fri Dec 09, 2011 7:04 am

Re: Revelation Test

Post by nbohr1more »

Good work.

I still hope someday to see SSAO as a modulation of the ambient lights rather than a post-process...
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Me2 :) Doom3 allready had a function for making a a copy of the depthrenderer but i refined it a bit.

Code: Select all

/*
====================
CopyDepthbuffer

This should just be part of copyFramebuffer once we have a proper image type field
====================
*/
void idImage::CopyDepthbuffer( int x, int y, int imageWidth, int imageHeight, bool useOversizedBuffer ) {
	this->Bind();
	// if the size isn't a power of 2, the image must be increased in size
	int	potWidth, potHeight;
	IMAGE_ROUND_POWER2( imageWidth, potWidth );
	IMAGE_ROUND_POWER2( imageHeight, potHeight );
	GetDownsize( imageWidth, imageHeight );
	GetDownsize( potWidth, potHeight );
	// Instruct openGL that we won't bind a color texture (ssao needs this).
	glReadBuffer( GL_NONE );
	// only resize if the current dimensions can't hold it at all,
	// otherwise subview renderings could thrash this
	if( ( useOversizedBuffer && ( uploadWidth < potWidth || uploadHeight < potHeight ) ) || ( !useOversizedBuffer && ( uploadWidth != potWidth || uploadHeight != potHeight ) ) ) {
		uploadWidth = potWidth;
		uploadHeight = potHeight;
		if( potWidth == imageWidth && potHeight == imageHeight ) {
			glCopyTexImage2D( GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, x, y, imageWidth, imageHeight, 0 );
		} else {
			glTexImage2D( GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT24, potWidth, potHeight, 0, GL_DEPTH_COMPONENT, GL_FLOAT, 0 );
			glCopyTexSubImage2D( GL_TEXTURE_2D, 0, 0, 0, x, y, imageWidth, imageHeight );
		}
	} else {
		// otherwise, just subimage upload it so that drivers can tell we are going to be changing
		// it and don't try and do a texture compression or some other silliness
		glCopyTexSubImage2D( GL_TEXTURE_2D, 0, 0, 0, x, y, imageWidth, imageHeight );
	}
	// if the image isn't a full power of two, duplicate an extra row and/or column to fix bilerps
	if( imageWidth != potWidth ) {
		glCopyTexSubImage2D( GL_TEXTURE_2D, 0, imageWidth, 0, x + imageWidth - 1, y, 1, imageHeight );
	}
	if( imageHeight != potHeight ) {
		glCopyTexSubImage2D( GL_TEXTURE_2D, 0, 0, imageHeight, x, y + imageHeight - 1, imageWidth, 1 );
	}
	glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );
	glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );
	glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );
	glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );
	backEnd.c_copyDepthBuffer++;
}
this one glReadBuffer( GL_NONE ); is mandatory to instruct opengl that we are not uploading a color texture.

i went as far as creating new functions for exposing this to the game dll's with a copy of CaptureRenderToImage and RB_CopyRender specialized for depthbuffer images but while it worked it broke compatibility with Doom3 so i removed it again for now.

the above function was used for the defunct shadow map code.
Productivity is a state of mind.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Ah i seem to have nailed why it would not work before i added a new enum but forgot that demos recorded by previous vanilla Doom3 needs those to be in a specific order so ->

Code: Select all

typedef enum {
	DC_BAD,
	DC_RENDERVIEW,
	DC_UPDATE_ENTITYDEF,
	DC_DELETE_ENTITYDEF,
	DC_UPDATE_LIGHTDEF,
	DC_DELETE_LIGHTDEF,
	DC_LOADMAP,
	DC_CROP_RENDER,
	DC_UNCROP_RENDER,
	DC_CAPTURE_RENDER,
	DC_END_FRAME,
	DC_DEFINE_MODEL,
	DC_SET_PORTAL_STATE,
	DC_UPDATE_SOUNDOCCLUSION,
	DC_GUI_MODEL,
	// new tokens need to come after this.
	DC_CAPTURE_DEPTH
} demoCommand_t;
and it works now :)

i also created new functions to forward the copy of the depthbuffer to the game dll's

in RB_STD_FillDepthBuffer

after

Code: Select all

	// if we are just doing 2D rendering, no need to fill the depth buffer
	if( !backEnd.viewDef->viewEntitys ) {
		return;
	}
yank this in

Code: Select all

	// copy of depthbuffer revelator.
	globalImages->currentDepthImage->CopyDepthbuffer( 
	backEnd.viewDef->viewport.x1,
	backEnd.viewDef->viewport.y1,
	backEnd.viewDef->viewport.x2 -
	backEnd.viewDef->viewport.x1 + 1,
	backEnd.viewDef->viewport.y2 -
	backEnd.viewDef->viewport.y1 + 1, true );
in tr.backend.cpp yank this in

Code: Select all

/*
=============
RB_CopyRender

Copy part of the current framebuffer to an image
=============
*/
const void	RB_CopyRender( const void *data ) {
	const copyRenderCommand_t	*cmd;
	cmd = ( const copyRenderCommand_t * )data;
	if( r_skipCopyTexture.GetBool() ) {
		return;
	}
	if( cmd->image ) {
		cmd->image->CopyFramebuffer( cmd->x, cmd->y, cmd->imageWidth, cmd->imageHeight, false );
	}
}
just before the const void RB_CopyDepth( const void *data ) function.

in rendersystem.cpp yank this in

Code: Select all

/*
================
CaptureDepthToImage
================
*/
void idRenderSystemLocal::CaptureDepthToImage( const char *imageName ) {
	if( !glConfig.isInitialized ) {
		return;
	}
	guiModel->EmitFullScreen();
	guiModel->Clear();
	if( session->writeDemo ) {
		session->writeDemo->WriteInt( DS_RENDER );
		session->writeDemo->WriteInt( DC_CAPTURE_DEPTH );
		session->writeDemo->WriteHashString( imageName );
		if( r_showDemo.GetBool() ) {
			common->Printf( "write DC_CAPTURE_DEPTH: %s\n", imageName );
		}
	}
	// look up the image before we create the render command, because it
	// may need to sync to create the image
	idImage				*image = globalImages->ImageFromFile( imageName, TF_DEFAULT, true, TR_REPEAT, TD_HIGH_QUALITY );
	renderCrop_t		*rc = &renderCrops[currentRenderCrop];
	copyRenderCommand_t *cmd = ( copyRenderCommand_t * )R_GetCommandBuffer( sizeof( *cmd ) );
	cmd->commandId = RC_COPY_DEPTH;
	cmd->x = rc->x;
	cmd->y = rc->y;
	cmd->imageWidth = rc->width;
	cmd->imageHeight = rc->height;
	cmd->image = image;
	guiModel->Clear();
}
just before the CaptureRenderToFile function.

in renderworld_demo.cpp in the idRenderWorldLocal::ProcessDemoCommand function

add this

Code: Select all

	case DC_CAPTURE_DEPTH:
		if( r_showDemo.GetBool() ) {
			common->Printf( "DC_CAPTURE_DEPTH\n" );
		}
		renderSystem->CaptureDepthToImage( readDemo->ReadHashString() );
		break;
just below

Code: Select all

	case DC_CAPTURE_RENDER:
		if( r_showDemo.GetBool() ) {
			common->Printf( "DC_CAPTURE_RENDER\n" );
		}
		renderSystem->CaptureRenderToImage( readDemo->ReadHashString() );
		break;
and in tr.bavkend.cpp in the RB_ExecuteBackEndCommands function

make it look like this

Code: Select all

void RB_ExecuteBackEndCommands( const emptyCommand_t *cmds ) {
	// r_debugRenderToTexture
	int	c_draw3d = 0, c_draw2d = 0, c_setBuffers = 0, c_swapBuffers = 0, c_copyRenders = 0, c_copyDepth = 0;
	if( cmds->commandId == RC_NOP && !cmds->next ) {
		return;
	}
	backEndStartTime = Sys_Milliseconds();
	// needed for editor rendering
	RB_SetDefaultGLState();
	// upload any image loads that have completed
	globalImages->CompleteBackgroundImageLoads();
	for( /**/; cmds ; cmds = ( const emptyCommand_t * )cmds->next ) {
		switch( cmds->commandId ) {
		case RC_NOP:
			break;
		case RC_DRAW_VIEW:
			RB_DrawView( cmds );
			if( ( ( const drawSurfsCommand_t * )cmds )->viewDef->viewEntitys ) {
				c_draw3d++;
			} else {
				c_draw2d++;
			}
			break;
		case RC_SET_BUFFER:
			RB_SetBuffer( cmds );
			c_setBuffers++;
			break;
		case RC_SWAP_BUFFERS:
			RB_SwapBuffers( cmds );
			c_swapBuffers++;
			break;
		case RC_COPY_RENDER:
			RB_CopyRender( cmds );
			c_copyRenders++;
			break;
		case RC_COPY_DEPTH:
			RB_CopyDepth( cmds );
			c_copyDepth++;
			break;
		default:
			common->Error( "RB_ExecuteBackEndCommands: bad commandId" );
			break;
		}
	}
	// go back to the default texture so the editor doesn't mess up a bound image
	glBindTexture( GL_TEXTURE_2D, 0 );
	backEnd.glState.tmu[0].current2DMap = -1;
	// stop rendering on this thread
	backEndFinishTime = Sys_Milliseconds();
	backEnd.pc.msec = backEndFinishTime - backEndStartTime;
	if( r_debugRenderToTexture.GetInteger() == 1 ) {
		common->Printf( "3d: %i, 2d: %i, SetBuf: %i, SwpBuf: %i, CpyRenders: %i, CpyDepth: %i, CpyFrameBuf: %i, CpyDepthBuf: %i\n", c_draw3d, c_draw2d, c_setBuffers, c_swapBuffers, c_copyRenders, c_copyDepth, backEnd.c_copyFrameBuffer, backEnd.c_copyDepthBuffer );
		backEnd.c_copyFrameBuffer = 0;
		backEnd.c_copyDepthBuffer = 0;
	}
}
in tr_local.h in the backEndState_t struct

aff this at the bottom
int c_copyDepthBuffer;
just after
int c_copyFrameBuffer;

and back in tr_local.h

Code: Select all

typedef enum {
	RC_NOP,
	RC_DRAW_VIEW,
	RC_SET_BUFFER,
	RC_COPY_RENDER,
	RC_COPY_DEPTH,		// added for depthbuffer copying
	RC_SWAP_BUFFERS		// can't just assume swap at end of list because of forced list submission before syncs
} renderCommand_t;
make the enum look like this.

viola you can now copy the Z buffer for ssao softshadows etv. :)
sadly sikkpins hacked version does not work to well with this as it was made for A vanilla Doom3 without these functions.

sikkpins depthrender is in playerview.cpp in the function idPlayerView::RenderDepth and used
renderSystem->CaptureRenderToImage( "_depth" ); for ssao and
renderSystem->CaptureRenderToImage( "_ssDepth" ); for softshadows.

in case he had not created those shaders with a crippled vanilla in mind it should have been

renderSystem->CaptureDepthToImage( "_depth" ); for ssao and
renderSystem->CaptureDepthToImage( "_ssDepth" ); for softshadows.

but it looks rather weird if you try with that considering the above.
Productivity is a state of mind.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Image

ssao in grimm quest ;)

turned out you need to make copy of both the framebuffer and the depthbuffer.

nasty shit this :S i also need to filter out viewentities and subviews so that only the world gets it, anyone have a pointer how to do this from the game dll ?
Productivity is a state of mind.
nbohr1more
Posts: 54
Joined: Fri Dec 09, 2011 7:04 am

Re: Revelation Test

Post by nbohr1more »

I wish I could help there. I know that Rebb helped Sikkpin with a similar request for his soft-shadows.

Did you see this?

https://github.com/raynorpat/dhewm3

(deferred renderer)
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Revelation Test

Post by revelator »

Huh interresting :shock: looks like raynor ditched his previous project and went for dhewm modified with a defered renderer.
Productivity is a state of mind.
Post Reply