NQ connecting to QW?

[ArchAngel]

I know some people have added NQ support into QW and I see DarkPlaces supports QW, but how much difference is there between the NQ and QW protocol. I know it's not a trivial matter to allow NQ engines access to QuakeWorld, but has anyone looked into what's needed to get things up and running?

[mh]

I tried doing a diff once but gave up after seeing some of the client changes. I've a hunch that it might be easier to go in the other direction (add NQ server support to the QW client); Spike would be able to give more info.

[Spike]

been there, done that, got the proxy. :)
fteqtv on fte's svn. You could probably turn that into a dll and wire it in to an NQ engine by wrapping the socket calls if you don't like separate binaries (aye, they're painful).

first thing is the connectionless protocols. the qw connection process is just in every single way better. No DoSing of servers by just spamming a few connect requests every now and then. No firewall issues either, so even if you don't want full QW support, this is worthwhile anyway - DP actually uses a very qw-like connectionless protocol, and its the basis of the q2 and q3 connection process, which are more pretty still.
Frankly, NQ's pre-connection messages are massively massively flawed.
(this can be found hidden somewhere in qw.c)

The next step is to support the netchan itself. NQ sends reliables and datagrams in separate packets. QW sends them in combined packets. The reliables are a bit fiddly, and stuff.
While you're doing that, you'll notice that the protocol is limited to only about 1450 bytes per packet, or 1024 or so for the reliable or unreliable sections on its own.
If you're doing just the client, then the server does all the fragmentation/backbuffering and the client doesn't really need to care, except for the aspect mentioned in the next paragraph.
(this can be found in qtv's netchan.c - qtv naturally has code for either qw or nq, inherited from fte, which might help reduce the 10 or so net_foo.c files in nq to just 1 or 2 if you're in the mood for large changes)

If you just want MVDs, you can ignore the above.
Because of those limits in the past paragraph, you need to support splitting the svc_serverinfo into sections, this is generally easy enough, but can result in certain edge cases which result in issues.

Okay, so you changed the connection process to use challenges instead, you changed how the client communicates with the server, and you added the qw svc_serverdata/svc_modellist/svc_soundlist svcs.
Now you need to add support for the delta protocols. The lame way is to never acknowlege them and then you get a complete set of data every single frame - and the server will choke your bandwidth. The proper way is to remember the past 64 frames, with max 64 ents per frame (engine extensions and mvds mean you should try and keep the 64 ents-per-frame-max unlimited).

You need to add support for svc_playerinfo to handle the player/non-player separation in quakeworld.

With that done, its possible that you'll be able to connect and stay on the server... But beware! there are more changes. svc_nails needs to be added for older servers. You need to handle muzzleflashes, you need to duplicate svc_updatestat to handle both 32bit and 8bit forms. you need to change the tempentity svcs, the sound svc, the svc_cdtrack svc got a byte removed too.
(the full svc differences can generally be seen in qtv's parse.c)

Actually, there's a lot of svcs. They're mostly small changes, but there are some big ones too. For the most part they're directly equivelent to older functionality.

I've not yet mentioned the clc changes. They should be simple enough to add, tbh, just make sure you throttle the clc_move generation rate to the server's cap, or it'll kick you. Each clc_move dupicates the last 2 frames as well as containing the new one.

Prediction is an extra. It depends upon the sequence numbers. For best results, copy an existing pmove.c rather than rewriting it - writing your own will result in inaccuracies/glitches unless it matches exactly.

There are a lot of changes all over the place, including stuff like player skins, userinfo, serverinfo, and stuff. You don't have those in NQ, so full support is not just protocol+prediction - you also need to understand enough stuffcmds properly to at least emulate them.

[ArchAngel]

Ok fantastic thanks, that's a lot to digest. Unfortunately simply linking the DLL isn't really an option for me as I'm working from the SharpQuake code base this time (C# all the way...), but at least having a timeline for the connection and the ability to partially test as I go a long will help. I'll have a good read through that again tomorrow at a saner hour and see where I get to...

... really quite tempted to have a look at porting a QW server over to Azure as well just for kicks

[Baker]

It is always cool when someone with past Quake interests has that interest revived.

If you don't mind, what are you engine goals?

I'm kinda curious since most people with an engine project have something different in mind. In recent times, most of them tend to be targeting another platform (PSP, iPhone, another gaming device, etc.) to some degree even with MH (Direct 3D ... although in addition to that, MH wreaks vengeance on the plethora of things he "does not like" or thinks should not be accepted) or Quakespasm (getting FitzQuake-style immersive single player in a desktop-platform neutral way) or well Mankrip/Qbism/leileilol+andrewj's software renderer stuffs.

[ArchAngel]

No, I don't mind at all

I'm working on 3 different Quake projects, because I've never been able to start a single project and stick to it... I get distracted and want to test out other things while I try and figure out why the other bits don't work properly

The first is an engine based on SharpQuake (Quake re-written in C#). It's great because of all the syntactic sugar in C#, it means a lot of the code can be made neater and I can use a lot of existing C# libs to extend things without too much hassle (for example, I got compressed Zip/PK3 support in the engine in around 10 minutes).
This project is designed NOT to be Quake... by which I mean I can break protocols and mess around with things that otherwise should be left the hell alone. One of the issues I've seen in Quake development in the 12 years since the source released is that coders keep finding themselves constrained by not wanting to break compatability with the original protocol, if you stop thinking about it as a Quake engine and just think of it as an engine it means you can play with whatever you feel like.

The second is a QuakeWorld client written in normal boring C. All I'm looking to do with this one is add some convienience to the engine without game play changing features. I want stuff like built-in server browsers and AVI recording, but I don't want transparent water or crazy shadows. While I love this stuff for normal play when it comes to QW that's all I want to play.

The third is a QuakeWorld server but re-written in C#. I'm planning on butchering some of SharpQuake to get a leg up on this project as it's worryingly huge. The final end goal with this is to port it to Windows Azure. As a Microsoft Partner I get free time and bandwidth with Azure, so I thought it might be cool to try out Azure with this, especially if I can get it to take advantage of some of the dynamic scaling from Azure.

[Baker]

The first is an engine based on SharpQuake (Quake re-written in C#). It's great because of all the syntactic sugar in C#, it means a lot of the code can be made neater and I can use a lot of existing C# libs to extend things without too much hassle (for example, I got compressed Zip/PK3 support in the engine in around 10 minutes).

Understandable for sure.

This project is designed NOT to be Quake... by which I mean I can break protocols and mess around with things that otherwise should be left the hell alone. One of the issues I've seen in Quake development in the 12 years since the source released is that coders keep finding themselves constrained by not wanting to break compatability with the original protocol, if you stop thinking about it as a Quake engine and just think of it as an engine it means you can play with whatever you feel like.

I think the engine route is more interesting than the Quake engine route.

I want to do things with my engine eventually that aren't Quake at all and maybe not even 3D (always) and may or may not even contain all of the components [I don't have any set "goals", I'm more curiosity oriented]. Yet I feel I can't drift there until I at a minimum like my engine as a Quake engine. I have a stripped down shell of my engine which I can do 3D experiments with OpenGL which helps with learning/understanding OpenGL better.

One reason to support Quake or Quakeworld "as-is" is to get better testing and feedback because you have a potentially larger audience pool.

And one danger of the "engine" route versus "Quake engine" is lack of standards. Now, obviously in that situation the "standards" don't matter because whatever you decide is the standard, yet in some ways "standards" are beneficial because they have been tested in practice and have had enough of a feedback loop and thinking behind them.

I will say it sounds like you have a tougher project. GLQuake is a bit of bugfest and also it isn't "easy" to convert GLQuake into GLQuakeWorld.

Anyways, just raw thoughts. I think it is cool you are pursing the "engine" route. I hadn't heard of SharpQuake before this thread. I hope you project goes well