[SharpQuake] - Compressed PK3 support

[ArchAngel]

Objective
=========

In this tutorial we're going to add Compressed PK3 support to SharpQuake.


Impetus
=========

PAK files are pretty horrible, you need special tools to add to them and they don't provide any form of compression. This will allow us to easily add files in an industry standard format.


Drawbacks
=========

This tutorial removes PAK support from the engine, but with appropriate checking it would be possible to add both if you so desired.

File compression always adds overhead to the engine, however any computer sufficiently modern to cope with the .NET framework shouldn't experience any real problems with PK3 support.


Pre-requisites
==========

SharpQuake - http://sourceforge.net/projects/sharpquake/
DotNetZipLibrary - http://dotnetzip.codeplex.com/
Visual Studio -or- Visual C# Express

Method
==========

First add a reference to the DotNetZip-Reduced library to our project. (Project Menu -> Add Reference -> Browse)

Open Common.cs

Find:

Code: Select all

using System.IO;

Add directly underneath it:

Code: Select all

using Ionic.Zip;

Next find the method - public static pack_t LoadPackFile(string packfile)
and completly remove the method.

Paste in the following method instead -

Code: Select all

//EGL PK3 Support
public static ZipFile LoadCompressedContainer(string filename)
{
  ZipFile output=null;
  try
  {
     output = ZipFile.Read(filename);
   }
   catch(Exception)
   {}
   return output;
}

Next find -

Code: Select all

 class searchpath_t

and replace it with this -

Code: Select all

class searchpath_t
{
public string filename; // char[MAX_OSPATH];
public ZipFile pack;          // only one of filename / pack will be used

public searchpath_t(string path)
{
  if (path.EndsWith(".pk3"))
  {
    this.pack = Common.LoadCompressedContainer(path);
    if (this.pack == null)
      Sys.Error("Couldn't load packfile: {0}", path);
  }
  else
    this.filename = path;
}

public searchpath_t(ZipFile pak)
{
  this.pack = pak;
}
} // searchpath_t;

This new class lets us search for PK3 files and if we find one open a handle to it via the LoadCompressedContainer method.

Next find the Path_f method and locate the line -

Code: Select all

Con.Print("{0} ({1} files)\n", sp.pack.filename, sp.pack.files.Length);

This needs to be changed to

Code: Select all

Con.Print("{0} ({1} files))\n", sp.pack.Name, sp.pack.Count);  //EGL: PK3

Next we need to find the - AddGameDirectory method

And change the line that reads -

Code: Select all

string pakfile = String.Format("{0}/pak{1}.pak", dir, i);

to read

Code: Select all

string pakfile = String.Format("{0}/pak{1}.pk3", dir, i);

Next we need to locate the - FindFile method

in there find the line that reads

Code: Select all

if(sp.pack != null)

You can now replace everything from there to closing curly brace with the following

Code: Select all

{
 // look through all the pak file elements
 ZipFile pak = sp.pack;

 if (pak.ContainsEntry(filename))
 {
   Stream entryStream = new MemoryStream();
   pak[filename].Extract(entryStream);
   file = new DisposableWrapper<BinaryReader>(new BinaryReader(entryStream, Encoding.ASCII), true);

   file.Object.BaseStream.Seek(0, SeekOrigin.Begin);

   return (int)entryStream.Length;
   }
}


All we're doing here is to create a MemoryStream into which we can extract the filename from the PK3 file. Everything else in this code works pretty much identically to the original code.

And we're done. Your next step is to extract the orignal PAK files and repackage them as ZIP files. The DotNetZipLibrary does all the donkey work for us here (which is always nice). It's worth noting that although .NET does support both ZIP compression and the ZIP file container, it doesn't support the ZIP container format in a standard way. The .NET framework version requires that all ZIP file has an XML manifest within them which lists the contents and thier MIME types, without this it simply see the ZIP files as empty... which is pretty damn irritating.

[LordHavoc]

It's not that hard to parse the zip directory yourself, you can find code for this in fs.c in darkplaces...

Requiring an xml manifest sounds absurd

[revelator]

Requiring an xml manifest sounds absurd

pretty much the reason i ditched msvc since the only versions of it supporting win7/vista all require manifests (YUCK) .

hmm i wonder if mono (gnu sharp compiler) could compile sharpquake ? that way we could ditch manifest dependancy.

[mh]

You can use the shell namespace to get programmable access to Windows "compressed folders" and interact with zip files that way. I have code somewhere for creating a zip file which might be helpful.

[andrewj]

I have code somewhere for creating a zip file which might be helpful.

If it's cross platform then I'd find that very useful!

[ArchAngel]

If it's cross platform then I'd find that very useful!

If it's from the shell namespace then it's very heavily non platform neutral I'm afraid. In fairness I try and avoid any of the shell namespace stuff especially from a managed environment. It's all a bit of a kludge and the MS implementation of the Zip stuff is pretty nasty.