Describing the Quake Command System

[frag.machine]

QF's completion isn't very smart yet. It certainly doesn't complete parameters, and I'm pretty sure it doesn't work after semicolons. It does, however, provide a list of all possible cvars and commands (including aliases?) that match the current "head". Certainly less annoying that id's very basic version.

Re parsing: may I suggest QF's plist code? It's actually quite self-contained (only dependencies are dstrings and hash tables, and all three modules are documented). There are some dependencies on sys (Sys_Printf etc) and mathlib/compat (min/max and strequal), but they're trivial to replace.

Property lists (include/QF/qfplist.h and libs/util/qfplist.c) (format docs)
Hash tables (include/QF/hash.h and libs/util/hash.c)
Dynamic Strings (include/QF/dstring.h and libs/util/dstring.c)

Yet about parsing: may I suggest JSON ? widely used, a de facto industry standard, very similar (but more complete) to the key/value approach you guys are aiming without the XML verbosity/complexity, supported among a great number of languages (including but not limited to C, C++, C#, Java, JavaScript - this one natively-, Perl, Python, etc.), with code to write and parse the format ready to use.

[mh]

Threading would work OK in the menus where it would show up as some latency in the info becoming visible rather than causing a hitch, but I suspect that it would be equally as annoying in the console. Press TAB and nothing happens for a short while. Unfortunately the console is the place where autocompletion is really needed, and equally as unfortunately I think that filtering out ammo box/etc models from the maps list is sufficiently important that maps do need to be opened and validated (the fact that the "map" command is the one where autocompletion is of most benefit just makes this suck even more).

So much for on-demand parsing; threading the scan at game change time may be of benefit but disk IO does remain the major bottleneck.

[Baker]

Threading would work OK in the menus where it would show up as some latency in the info becoming visible rather than causing a hitch, but I suspect that it would be equally as annoying in the console. Press TAB and nothing happens for a short while. Unfortunately the console is the place where autocompletion is really needed, and equally as unfortunately I think that filtering out ammo box/etc models from the maps list is sufficiently important that maps do need to be opened and validated (the fact that the "map" command is the one where autocompletion is of most benefit just makes this suck even more).

So much for on-demand parsing; threading the scan at game change time may be of benefit but disk IO does remain the major bottleneck.

I know you are in to perfect solutions. What about the opposite of your solution?

Don't scan the maps. Scan the progs.dat. If the .bsp is in the progs.dat, it isn't really a map. That would scan just 1 file.

Asses kicked, gum chewed ... and be living the good life in problems-minus-one-ville. And if I'm making a mod so I have miscellaneous .bsp models sitting around and I'm being stupid and think I can really get a quality game from playing chair.bsp or healthbox.bsp, that's just a case of the tough-shits since that isn't really an end-user problem ... ya know.

[mh]

Won't work with multiplayer though, where there may be a different progs on a different machine.

Not really bothered about it being a perfect solution; sensible with some imperfections is OK.

[Baker]

The .bsp model problem sucks.

[Spike]

if it has a leading b_ then ignore it?

scanning the progs won't work. most list the maps/foo.bsp files within the 'main' function. and those that don't still have the map names hardcoded in the intermission/finale code.

[taniwha]

A bit of a tangent, but on looking at QF's command and cvar code (with a mind to do some more work on completion), it turns out QF still maintains the alpha-sorted linked lists (however, hash tables most definitely are used for lookup by name).

[Baker]

Command parser

Code: Select all

/*
////////////////////////////////////////////////////////////////////////////////
Parser
////////////////////////////////////////////////////////////////////////////////
*/

#define MAX_LINE_LEN_1024	1024
#define MAX_NUM_ARGS_128	128
#define MAX_ARG_LEN_64		64

typedef struct
{
	text64	args[MAX_NUM_ARGS_128];
	int		count;
} line_parse_t;

line_parse_t commandargs;

typedef struct
{
	int		start;
	int		end;
	int		len;
} arg_t;

int Interpreter_Parse (line_parse_t* lineParse, const char* text)
{
	static const int maxlen63 = MAX_ARG_LEN_64 - 1;
	arg_t	argz[MAX_NUM_ARGS_128];
	arg_t*	arg = NULL;
	const char* cursor = text;
	int		i, count = 0, len = strlen(text);
	fbool	in_quote = False;
	memset (lineParse, 0, sizeof(*lineParse));

	// Locate split points
	for (i = 0; i < len; i ++, cursor++)
	{
		// Whitespace inside arg delimits, except if quoted
		if (*cursor <= ' ' && arg && in_quote == False )	
			arg = NULL;
		
		// Whitespace outside arg = ignore
		else if (*cursor <= ' ' && arg == NULL)	
			continue;
		
		// Comment terminates line
		else if (*cursor == '/' && cursor[1] == '/') 
			break;

		 // Quote toggles in_quote, then ignored
		else if (*cursor == '\"')
			in_quote = !in_quote;

		else if (arg) // Data .. if in_arg extend it
		{
			if (len == maxlen63)
				continue; // At limit, no room left
			arg->end ++;
			arg->len ++;
		}
		else // Data, wasn't in an arg so start one
		{
			if (count == MAX_NUM_ARGS_128)
				break; // Can't add more
			arg = &argz[count];
			count ++;
			arg->start = arg->end = i;
			arg->len = 1;
		}
	}

	for (i = 0; i < count; i ++)
		strncpyz (lineParse->args[i], &text[argz[i].start], argz[i].len + 1); // +1 = room for null term

	return (lineParse->count = count);
}

databuffer_t command_buffer;

void Interpreter_AddText (const char* text)
{
	size_t	numbytes		= strlen (text);
	
	Databuffer_AddBytes (&command_buffer, text, numbytes);
	Databuffer_AddBytes (&command_buffer, "\n", 1); // Command buffer needs newline terminated text
}


// Issue commands for everything we have until there is nothing left to process
void Interpreter_Run (void)
{
	char current_line[MAX_LINE_LEN_1024];
	static size_t maxbytes = sizeof(current_line);

	if (command_buffer.maxsize == 0)
		Host_FatalError ("Interpreter_Run:  No command buffer.  Not initialized");

	// Run until nothing left ...
	while (DataBuffer_GetTextLine_And_Remove (&command_buffer, current_line, maxbytes))
	{
		// Parse the line.
		if (Interpreter_Parse (&commandargs, current_line))
		{
			// Run the command
			Command_Execute (commandargs.args[0]);
		}
	}


}

fbool Interpreter_Init (void)
{
#define COMMAND_BUFFER_SIZE_65535 65535 // The buffer will add 1 byte of padding to get nice 65536.
	Databuffer_Alloc	(&command_buffer, COMMAND_BUFFER_SIZE_65535);
	return True;

}

void Interpreter_Shutdown (void)
{
	Databuffer_Dealloc	(&command_buffer);

}

[Baker]

(the fact that the "map" command is the one where autocompletion is of most benefit just makes this suck even more).

They should have done the folders differently in the Quake paks + progs.dat. And the health boxes should have been in thrown in progs folder. Looking back ...

[Spike]

they should have put sounds in the progs directory too, it makes about as much sense. and yes, separate complaint.

[mh]

Lotsa things only come through with hindsight. That's part of what makes Q2 and Q3A better platforms than Q1.

[Baker]

Yet about parsing: may I suggest JSON ? widely used, a de facto industry standard, very similar (but more complete) to the key/value approach you guys are aiming without the XML verbosity/complexity, supported among a great number of languages (including but not limited to C, C++, C#, Java, JavaScript - this one natively-, Perl, Python, etc.), with code to write and parse the format ready to use.

I think I'm going to drift into that general direction, especially since the format is ok.

I've modified the parser I'm playing with as such

Code: Select all

#ifdef REPEAT_FUNCTIONALITY
// Check for repeat command.  An unquoted trailing "{" starts it, a singular unquoted "}" ends it.
	if (count >= 2 && arg->quoted == False && text[arg->start] == '{' && arg->len == 1)
	{
		count --; // Strip the trailing "{"
		memset (&repeatCommand, 0, sizeof(repeatCommand));

		for (i = 0; i < count; i ++)
			strncpyz (repeatCommand.args[repeatCommand.count++], &text[argz[i].start], argz[i].len + 1); // +1 = room for null term

		in_repeat_command = &repeatCommand;
		return 0;
	}

	if (count == 1 && argz[0].quoted == False && text[argz[0].start] == '}' && argz[0].len == 1)
	{		
		in_repeat_command = NULL;
		return 0;		
	}

	if (in_repeat_command && count /* <--- don't act on blank lines */ )
	{
		memcpy (lineParse, &repeatCommand, sizeof(*lineParse) );
		
		// If we would overflow, cap the args
		if (lineParse->count + count > MAX_NUM_ARGS_128 - 1)
			count = (MAX_NUM_ARGS_128 - 1) - lineParse->count;
	}
#endif

And what this does is rather anti-climatic:

Code: Select all

msgbox {
	"red"
	"green"
	"blue"
}

But what I am really trying to build up to at the moment ...

Code: Select all

entities add {
// name origin angles
	"box1", {0, 15, 0}, {0, 0, 0}
	"box2", {0, 15, 0}, {0, 0, 0}
	"box3", {0, 15, 0}, {0, 0, 0}
}

[Baker]

Holy hell ...

The amount of fun you can start having playing around with building a "command line interpreter" is almost beyond description, after you get a few pain in the arse functions written.

I used to wonder what could possibly motivate someone to, say, work on QuakeC compilers. Or scripting. Which seems boring as hell. I get that now. The possibilities are endless ...

/ My initial interest in this really was some "robust" 3D menus and doing it in a flexible way, so I can't do that within the Quake entity system. Using parsing, not only can I largely define this stuff in a text file, but defining 2D interfaces with hotspots on the fly seems within range.

[Spike]

much of the joy of writing a compiler is writing code that writes code. good luck testing every single possible combination though.

[Baker]

much of the joy of writing a compiler is writing code that writes code. good luck testing every single possible combination though.

Tangents like these are often my achilles heel in a distraction sense, and I risk getting lost in this little (but awesome) world of the command line interpreter ...

That being said, I think this particular side-quest might be the most worthy one I've gotten lost on ... provided I'm fast, work quickly, stay within "limits" and don't get infinitely bogged down ...

Code: Select all

// 
// Entities defintion
// 

entities_fields name texture origin

entities_add {
// name, texture, origin
	
	"cube1"	"bars.tga" (25, 10, 11)
	"cube2"	"bars.tga" (25, 15, 11)
	"cube3"	"globe.tga" (20, 15, 11)
}

What I've done is create a command called "entities_fields" which let's me specify the arguments to entities_add. Now I create an "entities_default" command like "entities_default size 1" or "entities_default size (0.5, 1, 2)" or "entities_default reset" so I can add entities with minimal text and no need to explicitly set 75 entities fields.