Quake Memory Manager - Part 1: the Zone
Posted: Wed Jan 06, 2010 9:36 pm
I'm going to do a series of tutorials in which we tackle the most maligned of wee beasties, the Quake memory manager. This code will be based on standard Win/GL Quake and will only require changes to be made to the zone.c file. I'm maintaining the exact same interface as the original Quake memory manager so that anyone can easily use it in their own engines.
This code will be usable with both software and GLQuake, and in fact I'm developing and testing it on software Quake to ensure this. Where possible I'm writing it to be completely portable, although unfortunately the Hunk is most likely going to break this objective.
For part 1 we're going to tackle the Zone. This is the region of Quake's memory which is used for allocation of small strings and other didgeridoos, although at the end of this part you'll be able to use it for whatever you want, no matter how big.
Concept
By default the Zone is a very tight region (47 K) and shares it's memory space with the rest of Quake's memory. Sometimes Quake runs out of Zone space and crashes, so you need to restart with the -zone command-line option and replay up to the point where it crashed.
Here we're going to completely remove that limitation (the only limitation will be the address space of your OS) and also remove the Zone from the shared memory area (so that Zone corruptions don't corrupt elsewhere).
Code Changes
We're going to replace every single Zone function with standard C memory functions instead. I'll give them all here, and make comments where required.
We don't need to use this function anymore, but we'll retain it for compatibility with code that might be elsewhere. This is the general pattern I'm going to follow throughout this series.
Note the ninja-trickery in this one. I'll discuss what's happening here after I give Z_Malloc.
Right. What we're doing here is allocating an extra 4 bytes in addition to the requested size. The first 4 bytes of the buffer are set to the ZONEID value, which is used as a guard value in Z_Free. The buffer we return starts following those 4 bytes.
For Z_Free what we do is back up by 4 bytes from the address given (a pointer is just a memory address and you can go backwards from it as well as forwards), checking that the first 4 bytes equal ZONEID (in case we try to Z_Free something that wasn't allocated via Z_Malloc), then freeing it from that point.
This function was never actually used in the engine. I guess it might have been experimental QuakeII stuff as QuakeII does use Z_Malloc with tags.
Likewise we don't use these any more but we keep them to maintain a compatible interface.
Final code change is at the bottom of the Memory_Init function (go find it at the bottom of zone.c). See this block:
We don't need it anymore as we no longer have Zone limitations and we certainly don't want to pollute our main memory with something that is never used. Remove it. Delete it. Take it outside and kill it until it is dead.
Cleaning Up
When I'm finished I'll release a fully modified and cleaned up Zone.c, but for now you can go through the file, identify any variables or structures that are no longer used, and delete them.
The next part
For the next part I think we're going to do the Cache. The Cache is heavily dependent on Hunk allocations, so we're likewise going to break that dependency and put it in it's own memory space, giving it room to grow as large as it wants.
When this is posted (in a coupla hours time
) you'll have a very clean and tight zone.c, that looks virtually unrecognisable compared to the old one.
And then we'll do the Hunk.
This code will be usable with both software and GLQuake, and in fact I'm developing and testing it on software Quake to ensure this. Where possible I'm writing it to be completely portable, although unfortunately the Hunk is most likely going to break this objective.
For part 1 we're going to tackle the Zone. This is the region of Quake's memory which is used for allocation of small strings and other didgeridoos, although at the end of this part you'll be able to use it for whatever you want, no matter how big.
Concept
By default the Zone is a very tight region (47 K) and shares it's memory space with the rest of Quake's memory. Sometimes Quake runs out of Zone space and crashes, so you need to restart with the -zone command-line option and replay up to the point where it crashed.
Here we're going to completely remove that limitation (the only limitation will be the address space of your OS) and also remove the Zone from the shared memory area (so that Zone corruptions don't corrupt elsewhere).
Code Changes
We're going to replace every single Zone function with standard C memory functions instead. I'll give them all here, and make comments where required.
Code: Select all
void Z_ClearZone (memzone_t *zone, int size)
{
}Code: Select all
void Z_Free (void *ptr)
{
int *zblock = ((int *) ptr) - 1;
if (zblock[0] != ZONEID)
Sys_Error ("Z_Free: freed a pointer without ZONEID");
free (zblock);
}Code: Select all
void *Z_Malloc (int size)
{
int *zblock = (int *) malloc (size + sizeof (int));
if (!zblock)
Sys_Error ("Z_Malloc: failed on allocation of %i bytes", size);
memset (zblock, 0, size + sizeof (int));
zblock[0] = ZONEID;
return (zblock + 1);
}For Z_Free what we do is back up by 4 bytes from the address given (a pointer is just a memory address and you can go backwards from it as well as forwards), checking that the first 4 bytes equal ZONEID (in case we try to Z_Free something that wasn't allocated via Z_Malloc), then freeing it from that point.
Code: Select all
void *Z_TagMalloc (int size, int tag)
{
return Z_Malloc (size);
}Code: Select all
void Z_Print (memzone_t *zone)
{
}
void Z_CheckHeap (void)
{
}Final code change is at the bottom of the Memory_Init function (go find it at the bottom of zone.c). See this block:
Code: Select all
p = COM_CheckParm ("-zone");
if (p)
{
if (p < com_argc-1)
zonesize = Q_atoi (com_argv[p+1]) * 1024;
else
Sys_Error ("Memory_Init: you must specify a size in KB after -zone");
}
mainzone = Hunk_AllocName (zonesize, "zone" );
Z_ClearZone (mainzone, zonesize);Cleaning Up
When I'm finished I'll release a fully modified and cleaned up Zone.c, but for now you can go through the file, identify any variables or structures that are no longer used, and delete them.
The next part
For the next part I think we're going to do the Cache. The Cache is heavily dependent on Hunk allocations, so we're likewise going to break that dependency and put it in it's own memory space, giving it room to grow as large as it wants.
When this is posted (in a coupla hours time
) you'll have a very clean and tight zone.c, that looks virtually unrecognisable compared to the old one.And then we'll do the Hunk.
