Page 1 of 1

Mod_ClearAll -- doesn't always cut it

Posted: Mon Jul 28, 2014 11:39 pm
by Baker
The Horde of Zendar just might be #1 of all time in the quality of the environment --- It's scary because this map is truly unreal in detail.

It also has a special feature: it manages to bust many engines gamedir change (FitzQuake 0.85, Quakespasm, JoeQuake, Mark V, Qrack no doubt).

Here is how to create the problem: Start engine like normal (game = id1). Load E1M1, which will precache nail boxes and such.

Type "game zendar; map zendar" or "gamedir zendar; map zendar" in the console.

If you are lucky, the engine being used detects something afoul and you'll get a warning message or error. Other engines, it just crashes. It looks like old cache data for nail boxes, shellboxes and such is being used when the new ones are entirely different and this causes some invalid values in lightmap sizing to be used.

This is the code that Quake normal uses to clear mod data, and works without gamedir changes pretty reliably.

Code: Select all

/*
================
Host_ClearMemory

This clears all the memory used by both the client and server, but does
not reinitialize anything.
================
*/
void Host_ClearMemory (void)
{
	Con_DPrintf ("Clearing memory\n");
	Mod_ClearAll (); // <------------------------------------ Clear the models

	Hunk_FreeToLowMark (host_hunklevel);

	cls.signon = 0;
	memset (&sv, 0, sizeof(sv));
	memset (&cl, 0, sizeof(cl));
}
Which looks like this:

Code: Select all

/*
===================
Mod_ClearAll
===================
*/
void Mod_ClearAll (void)
{
	int		i;
	qmodel_t	*mod;

	for (i=0 , mod=mod_known ; i<mod_numknown ; i++, mod++)
		if (mod->type != mod_alias)
		{
			mod->needload = true;
			TexMgr_FreeTexturesForOwner (mod); //johnfitz // <----- Baker:  FitzQuake-derived engines have this
		}
}
But this isn't enough when a gamedir change happens.

I made a function that handles this thoroughly that can be used for a gamedir change:

Code: Select all

/*
===================
Mod_ClearAll_Compact
===================
*/
void Mod_ClearAll_Compact (void)
{
	size_t	bufsize = sizeof(qmodel_t) * MAX_MOD_KNOWN;
	qmodel_t*  new_mod_known = calloc (sizeof(qmodel_t), MAX_MOD_KNOWN );
	int		i, newcount = 0;
	qmodel_t	*mod;

	for (i=0 , mod=mod_known ; i<mod_numknown ; i++, mod++)
	{
		if (mod->type != mod_alias)
		{
			mod->needload = true;
			TexMgr_FreeTexturesForOwner (mod); //johnfitz
		}
		else
		{
			qmodel_t	*old_mod = &mod_known[i];
			qmodel_t	*new_mod = &new_mod_known[newcount];
			// Transfer it
			memcpy (new_mod, old_mod, sizeof(qmodel_t));
			newcount ++;
		}
	}

	memset (mod_known, 0, sizeof(mod_known));
	memcpy (mod_known, new_mod_known, sizeof(mod_known));

	mod_numknown = newcount;
	free (new_mod_known);
}
Calling the above function after a gamedir change should resolve any issues. I do this upon gamedir change:

Code: Select all

	//clear out and reload appropriate data
	Cache_Flush ();
	Mod_ClearAll_Compact ();  // <------------ Baker, formerly just a Mod_ClearAll

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 5:01 pm
by szo
Isn't the following simpler and more effective:

Code: Select all

void Mod_ResetAll (void)
{
	int		i;
	qmodel_t	*mod;

	for (i=0 , mod=mod_known ; i<mod_numknown ; i++, mod++)
	{
		if (!mod->needload) //otherwise Mod_ClearAll() did it already
			TexMgr_FreeTexturesForOwner (mod);
		memset(mod, 0, sizeof(qmodel_t));
	}
	mod_numknown = 0;
}
... and call the new Mod_ResetAll() just before that CacheFlush() ?

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 5:09 pm
by szo
szo wrote:... and call the new Mod_ResetAll() just before that CacheFlush() ?
I meant just _after_ CacheFlush(), sigh...

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 5:17 pm
by Spike
nah, you've gotta do it properly...
shell(va("\"%s\" -game %s %s", cmd_argv[0], newgamedir, cmd_args));
exit(1);

resolves a few other edge cases too! just make sure you don't do weird stuff with dupe -games.


reordering models randomly sounds like a buggy idea, especially if there are any stale pointers to various models lying around, like static entities (which annoyingly get redrawn on loading screens) and stuff like that...

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 7:06 pm
by r00k
DirectQ does a nice job of clearing out and reloading everything.

windows uses the 'system' command instead of 'shell' i tried to do this rebuilding the commandline but the task starts, yet the old task wont exit... :| wait unless its because im debugging...

Code: Select all

void xxxCOM_SetGameDir (char *dir)
{
	int		i, s;
	char	*text = NULL;

	s = 0;
	for (i=1 ; i<com_argc ; i++)
	{
		if (!com_argv[i])
			continue;
		s += strlen (com_argv[i]) + 1;
	}
	if (!s)
		return;

	text = Z_Malloc (s + 1);

	text[0] = 0;

	for (i=1 ; i<com_argc ; i++)
	{
		if (!com_argv[i])
			continue;
		strcat (text, com_argv[i]);
		if (i != com_argc-1)
			strcat (text, " ");
	}
	Q_snprintfz (com_gamedir, sizeof(com_gamedir), "%s/glQrack.exe -game %s %s", com_basedir, dir, text);
	system(va("%s\n",com_gamedir));	
	Z_Free (text);
	exit(1);		

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 7:25 pm
by szo
@Spike:
Well, of course there is execve() and its frontends like execv(), but I really don't see the point.
Can you document what the other edge cases are?

@root:
Of course the old one would not exit if you spawn the new one with system()

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 7:43 pm
by r00k
whoops! ok trying another way :)

okay this works! Also fixes my issues with zendar too! I thought it was a bug in protocol 666 that i added so thanks to Baker for pointing this whole cache flaw and Spike for his suggestion! :D

Code: Select all

void COM_SetGameDir (char *dir)
{
	int		i, s;
	char	*text = NULL;
	STARTUPINFO si;
	PROCESS_INFORMATION pi;

	CL_Disconnect_f ();

	s = 0;
	for (i=1 ; i<com_argc ; i++)
	{
		if (!com_argv[i])
			continue;
		s += strlen (com_argv[i]) + 1;
	}

	if (s)
	{
		text = Z_Malloc (s + 1);

		text[0] = 0;

		for (i=1 ; i<com_argc ; i++)
		{
			if (!com_argv[i])
				continue;
			strcat (text, com_argv[i]);
			if (i != com_argc-1)
				strcat (text, " ");
		}
	}

	memset (&si, 0, sizeof(si));
	si.cb = sizeof(si);
	si.wShowWindow = SW_HIDE;
	si.dwFlags = STARTF_USESHOWWINDOW;

	Q_snprintfz (com_gamedir, sizeof(com_gamedir), "%s/glQrack.exe -game %s %s", com_basedir, dir, text);
		
	if (!CreateProcess(NULL, com_gamedir, NULL, NULL, FALSE, 0, NULL, com_basedir, &si, &pi))
	{
		Con_Printf ("Couldn't execute %s\n", com_gamedir);
		return;
	}
	
	Z_Free (text);
	exit(1);		
}

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 7:48 pm
by Spike
@szo:
default cvar values, etc. carrying config settings (cvars+binds) from one mod to another might be undesirable too. plus there's cached 2d artwork and sounds to worry about as well.

@rook:
spawning a new process to do this is completely evil (if only because you need to create a new connection to the server if the gamedir changes on map change, which some QW mods do). The point is that there's a lot more state than just models and filesystem - and not all of it needs to be purged.
It really is just a quick hacky way to purge everything - probably more than is desired. :P

you should use GetModuleFileName(NULL, path, sizeof(path)) to obtain the name of the exe. you should also ensure that things are quoted properly just in case there's a space in one of the arguments/paths.

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 7:54 pm
by r00k
Ya it does seem to have issues online :(

Re: Mod_ClearAll -- doesn't always cut it

Posted: Tue Jul 29, 2014 11:47 pm
by Baker
This bug used to drive me crazy.

I'd occasionally see it, I'd restart and try to get the bug again and never could reproduce it reliably.

@szo: Maybe setting mod_known to 0 works just as well, but the "mcache" command shows lots of empty stuff and I'll probably re-use the code for a better "mcache" display at some point.

Re: Mod_ClearAll -- doesn't always cut it

Posted: Wed Jul 30, 2014 9:41 am
by szo
Applied my patch to quakespasm repo at r942:
http://sourceforge.net/p/quakespasm/code/942/

Fixes the issue in quakespasm. Eric's build bot at
http://quakespasm.ericwa.com/job/quakespasm/ should
generate a new build soon.