Doom 3 engine release and game code

[revelator]

So far only problem i had using this was that the parralax shader from sikkmod wouldnt work (figures as the glsl backend when on, uses the internal interaction shader so he would have to redo that one to use glsl).
Else it looks pretty fantastic much clearer than the old ARB2 shader.

[mh]

And here's the full and final GPL-compatible version of everything: http://www.quaketastic.com/upload/files ... haders.zip

[motorsep]

does it have shadows and material parser ?

[mh]

Not very demanding, are we?

No - it's just the interaction shaders that are GLSL (hint - reading the readme would have told you that ). Everything else is still handled the same way as before.

[motorsep]

Not demanding, just asking

I thought complete GLSL backend would be with shadows and material parser Anyhow, are you planning on expanding it with GLSL shadows and material parser (the one that Spike linked) or just layed out the foundation for other coders (reckless maybe? ) to build upon it ?

Note that I am not demanding anything or whatnot, just trying to see where it's all heading

[mh]

I don't have any long term plans for the Doom 3 engine. This was just a quick 2 evenings work because it tickled my interest - if others want to build on it they can for sure, but I may or may not and it may be in one month's time or one year's time.

[nbohr1more]

This is cool work but don't be surprised if it isn't greeted with jubilation. Most folks were under the impression that Raynorpat and LogicalError (Sander Van Rossen!) already had 90% working GPL compliant GLSL backends and that the last hurdle
was the missing Glasswarp shaders and other material specific programs.

Don't let that (or demanding folks) discourage you from doing any more tinkering though. I am certainly interested no matter
the level of completion or outcome.

[revelator]

As far as im concerned im still going to try and get it complete support i cannot give any timeframe though as im just starting in C++.
Im getting a shader error in one of the shaders btw this one

layout(location = in vec4 TexCoord; unexpected ( should be :: something ?.
i also had to put this at the top #version 120 to kill of a warning about one of the functions only being avaliable with that version.

I remember Sikk had the same problem with his glsl shaders something about the syntax going out of whack.

I guess the same idea used on the interaction shaders could be used on the material parts ? going to try and see if its possible but if MH comes up with something im surely not going to complain

[revelator]

Small fix to the code

Code: Select all

static float *RB_GLSL_MakeMatrix (const float *in1, const float *in2, const float *in3 = NULL, const float *in4 = NULL)
{
   static float m[16]; // increased cause else the memory copy is to small

   if (in1) memcpy (&m[0], in1, sizeof (float) * 4);
   if (in2) memcpy (&m[4], in2, sizeof (float) * 4);
   if (in3) memcpy (&m[8], in3, sizeof (float) * 4);
   if (in4) memcpy (&m[12], in4, sizeof (float) * 4);

   return m;
}

still cannot get the GPL shader to work hmm :S it seems to use a mix of version 120 and 130 but im unable to force both. Old version works fine but im not to sure abut using them.

[mh]

Oops, clumsy of me. I had that originally set up for just handling the 2x4 diffuse/bump/specular matrixes, and made the change to more generic matrix handling quite late. At least that's what I'm telling my lawyer.

layout (location=n) can be omitted - you'd use an attribute location table in the C++ code instead and pass that as a param to the shader creation function, calling glBindAttribLocation based on the contents of the table. The only thing to be careful of is to make the glBindAttribLocation calls after the shaders have been attached but before the program has been linked. There's an example implementation of this in my first version (on pastebin, with the non-GPL-friendly shaders).

Alternatively you could scrap generic attrib arrays and use glTexCoordPointer calls instead (being careful to use glClientActiveTexture too - note that the stock code uses GL_SelectTextureNoClient so change that to GL_SelectTexture and specify the arrays following each Bind). I'd be personally more inclined to keep generic arrays though.

I think layout (location=n) is a GL3.x+ feature, but I wasn't able to find definite confirmation of which version it went core in; it's also available via GL_ARB_explicit_attrib_location on earlier versions. Non-square matrixes are GL2.1 and I don't think there's an extension for it. The lack of a GL_ARB_explicit_uniform_location is shocking and not being able to specify default values for sampler uniforms is also crap. GLSL's behaviour for both of these gives nothing of real value and just piles up the supporting infrastructure needed before you can get down to the fun part (actually writing shaders). HLSL's ": register(c0)" setup is infinitely superior in practice.

Laughing at the fact that AMD were happy without a #version but NV choked on it - it's normally the other way around. I guess NV have been tightening up on spec conformance lately.

It should be easy enough to extend this code to handle user-defined materials, but the material shader would need to be written in GLSL. When draw time comes check a flag on the material to determine it's type and send it through the appropriate path. The overhead of constantly enabling/disabling different shader modes may pile up though. By then you definitely need something like an RB_SelectShaderMode call that just selects the appropriate mode - the stock Doom 3 code will often disable shaders entirely at the end of one draw pass only to just re-enable them again at the start of the next one. The more you switch shader modes the more this will cause perf trouble, so better to cache the current mode and only switch (or enable/disable) if actually needed.

Doing an ARB to GLSL parser would be a lot more work, as would be implementing GLSL versions of the other ARB shaders, in particular because there are no alternate shader paths provided for them so there's nothing to derive a GPL-friendly version from. For ARB-to-GLSL the lack of explicit uniform locations and default sampler values in GLSL are particularly going to cause huge trouble - because ARB has both, so things would be a bit wild west with GLSL, and the resulting shader would need to be quite tightly coupled to the C++ code to work around it.

What seems more interesting is doing shader implementations of the remaining fixed-pipeline paths in the engine - the stuff in RB_STD_T_RenderShaderPasses should convert quite well to a single generic vertex/fragment shader pair.

As always with Doom 3, the turnaround time between initially writing code and actually getting to test it in the engine is incredibly long - lots of code, glacial compile times, slow loading don't make for easy and rapid iteration of changes. This makes wide-ranging changes more annoying than they otherwise would be.

[revelator]

Hehe no problem m8 with how quick you got this sampled together i wouldnt expect it to not have atleast one smal bugger

Doing the material shaders in glsl is ok with me the main reason im in for a glsl backend is that i need something that can do stuff that ARB2 cannot.
Like screen space ambient occlusion which might be slow (allthough some seem to disagree) but damn it looks good Sikk has a version which uses a few workarounds of not being able to access the depth buffer
by using a real image as a buffer but it has some drawbacks like fucking up skyboxes / heathaze (in fact everything that has more than one layer).

Gonna try and see if i can create something from your info but my head is spinning allready xD so it might take a while.

And aye nvidia does seem to have cleaned up spec conformance a lot

[mh]

Code: Select all

// one-time only
glTexImage2D (GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, width, height, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_BYTE, NULL);

// each frame
glCopyTexSubImage2D (GL_TEXTURE_2D, 0, 0, 0, x, y, width, height);


That'll get you the depth buffer. It's most likely a "doing it from the game code" limitation, not a technical limitation.

[revelator]

Could be aye going to see what Sikk thinks about it.

Btw heres some shaders that work but alas the vertex one is not GPL.
Also had to do a small fix.

interaction_fs.glsl

Code: Select all

#version 120

uniform vec4 u_light_origin;
uniform vec4 u_view_origin;
uniform vec4 u_color_modulate;
uniform vec4 u_color_add;
uniform mat2x4 u_diffMatrix;
uniform mat2x4 u_bumpMatrix;
uniform mat2x4 u_specMatrix;
uniform mat4x4 u_projMatrix;
uniform mat4x4 u_fallMatrix;
uniform sampler2D bumpImage;
uniform sampler2D lightFalloffImage;
uniform sampler2D lightImage;
uniform sampler2D diffuseImage;
uniform sampler2D specularImage;
uniform vec4 u_constant_diffuse;
uniform vec4 u_constant_specular;

varying vec2 diffCoords;
varying vec2 bumpCoords;
varying vec2 specCoords;
varying vec4 projCoords;
varying vec4 fallCoords;
varying vec3 lightDir;
varying vec3 halfAngle;
varying vec4 Color;

void main ()
{
   vec4 lightProject = texture2DProj (lightImage, projCoords);
   vec4 lightFalloff = texture2D (lightFalloffImage, vec2(fallCoords)); // hmm needed a cast here error otherwise.
   vec3 bumpMap = texture2D (bumpImage, bumpCoords).agb * 2.0 - 1.0;
   vec4 incomingLight = lightProject * lightFalloff;

   incomingLight *= max (dot (normalize (lightDir), bumpMap), 0.0);

   vec4 diffMap = texture2D (diffuseImage, diffCoords);
   vec4 specMap = texture2D (specularImage, specCoords);
   float specularPower = clamp ((max (dot (normalize (halfAngle), bumpMap), 0.0) - 0.75) * 4.0, 0.0, 1.0);

   specularPower *= specularPower;

   vec4 finalSpec = specularPower * specMap * 2.0 * u_constant_specular;
   vec4 finalDiff = diffMap * u_constant_diffuse;

   gl_FragColor = (finalDiff + finalSpec) * incomingLight * Color;
}



interaction_vs.glsl

Code: Select all

#version 120

in vec4 attrTexCoords;
in vec4 attrTangents0;
in vec4 attrTangents1;
in vec4 attrNormal;

mat3x3 u_lightMatrix = mat3x3 (attrTangents0, attrTangents1, attrNormal);

uniform vec4 u_light_origin;
uniform vec4 u_view_origin;
uniform vec4 u_color_modulate;
uniform vec4 u_color_add;
uniform mat2x4 u_diffMatrix;
uniform mat2x4 u_bumpMatrix;
uniform mat2x4 u_specMatrix;
uniform mat4x4 u_projMatrix;
uniform mat4x4 u_fallMatrix;
uniform sampler2D bumpImage;
uniform sampler2D lightFalloffImage;
uniform sampler2D lightImage;
uniform sampler2D diffuseImage;
uniform sampler2D specularImage;
uniform vec4 u_constant_diffuse;
uniform vec4 u_constant_specular;

varying vec2 diffCoords;
varying vec2 bumpCoords;
varying vec2 specCoords;
varying vec4 projCoords;
varying vec4 fallCoords;
varying vec3 lightDir;
varying vec3 halfAngle;
varying vec4 Color;

void main ()
{
   // we must use ftransform as Doom 3 needs invariant position
   gl_Position = ftransform ();

   diffCoords = attrTexCoords * u_diffMatrix;
   bumpCoords = attrTexCoords * u_bumpMatrix;
   specCoords = attrTexCoords * u_specMatrix;

   projCoords = gl_Vertex * u_projMatrix;
   fallCoords = gl_Vertex * u_fallMatrix;

   Color = (gl_Color * u_color_modulate) + u_color_add;

   vec3 OffsetViewOrigin = (u_view_origin - gl_Vertex).xyz;
   vec3 OffsetLightOrigin = (u_light_origin - gl_Vertex).xyz;
   
   lightDir = OffsetLightOrigin * u_lightMatrix;
   halfAngle = (normalize (OffsetViewOrigin) + normalize (OffsetLightOrigin)) * u_lightMatrix;
}


needed a cast in the lightfallof as the type copied was vec4 but the target was vec2

[revelator]

hmm ok seems im starting to figure out how it works.

the cast is not needed if we do fallCoords.xy it seems to work fine but im wondering a bit why its perfectly legal from the hardcoded version but not from the externally loaded one ?.

[mh]

Incidentally, the quality difference you noticed is most likely not some magical property of GLSL but instead due to "OPTION ARB_precision_hint_fastest;" in the original fragment program. NV gfx cards will run in two floating-point precision modes - 16-bit or 32-bit - and this option will bump them down to 16-bit. Hacking the program to use "OPTION ARB_precision_hint_nicest;" instead should give you 32-bit.

If you were using AMD you wouldn't have noticed this so much as AMD's lowest quality is 24-bit; older ones will only support 24-bit, newer will support 24-bit or 32-bit (on account of D3D10 and 11 requiring 32-bit throughout the pipeline, they kind of had to put in the support).