I found it a very interesting and informative test, and it's certainly highlighted a few areas where engine coders may need to concentrate their efforts.
I'm not going to get too hung up over what an engine I made 4 or 5 years ago can or cannot do, but I will make some general comments that I think apply across the board.
Mod support is always the killer. You never know what weird quirks a mod might exploit in order to do it's thing, especially if the QC and/or map sources are unavailable. Even something as basic as a cleaned up render path could end up breaking certain mods.
I'd love to see an "engine killer" map that only used ID1 features - that would be a much more valid and useful tool for an engine coder to build compatibility for larger maps. At least one could focus one's efforts on genuine support for the larger map, rather than continually getting sidetracked with building fixes for unintended QC exploits.
A lack of true standards is another thing that probably explains why a lot of engines don't have certain things implemented. Take loading a sky box from worldspawn. What key name do you use? What folder do the sky boxes go in? Does the value include the path or not? What's the naming convention for the 6 components? How do you react if you can't load all 6? Copying what Darkplaces does is the nearest we have to a standard, but then we get a new map that's designed for a specific faithful/minimalist engine that does things differently! The poor coder has to make the choice between trying to support every possible standard (and potentially creating a mess in the process), only implementing what seems reasonable to them (and potentially getting burned when their engine doesn't support stuff) or not bothering at all. It's a no-win situation.
The faithful/minimalist/purist people are another stumbling block. Engine coders
like adding new effects, because they
like Quake The Game but want to bring it's overall look up to more modern standards (or at least I know that's my motivation - when I see Doom 3 maps, I just think "wouldn't it be so cool if e2m5 looked at least partially like that?"). For most people this isn't a problem, but there is an
extremely vocal minority who shoot down anything that doesn't look like DOS Quake at 320 x 200 (OK, silly exaggeration, but I'm sure you know what I mean).
Finally, there is no true collaboration among engine coders. I'd personally
love to work with some of the folks out there on a single common codebase. Such a custom engine would stand a better chance of being all-things-to-all-people, and individual coders would be better able to focus on specific areas they're stronger in (in my case I'm good at kickstarting new and different ways of doing things, but a poor finisher). Collaboration is limited to exchanging snippets of code and giving the occasional helping hand, but it stops there. Most work is done in isolation. It's actually worse since QuakeSrc went down, as there's no melting pot of ideas any more.
It seems this got a bit more rant-ish than I'd intended, but I do think it's relevant in the context of this test, and would be interested in hearing what other engine coders think.
