Sending map data from server to client?

[andrewj]

I'm making a 2D game based on the Quake engine, and I need to send 2D map data (tile numbers) from the server to the client, since the maps will be randomly generated on the server.

What's a good way to do this?

At the moment I have it sending tile data in a similar way it sends visible entities (i.e. in SV_SendClientDatagram). But I think it might work better sending the whole map before game begins, during the signon stages. But how can you handle lost packets??

Cheers in advance.

[taniwha]

I don't know about quake*, but for quakeworld, signon packets are "reliable".

* My quake (as opposed to quakeworld) protocol knowledge is rather sketchy at best. I've spent much more time with the qw protocol.

[Spike]

both have some prespawn/signon/init buffers for things like static entities.
depends how much data there is. if the dataset is quite large, consider loading it on demand as the view moves around the map. just some function to get/request the data that's allowed to fail if its not available yet.

[taniwha]

Yeah, I know of the buffers. It's the reliable/unreliable part about which I'm uncertain.

As an aside, I gave QF's qw-server unlimited signon buffers yesterday

[Spike]

meanwhile I'm trying to deprecate the signon buffers in FTE (signon buffers only work if all clients use the same protocol).
MSG_INIT is always an issue.
but that's offtopic.

Anyway, it depends how big your map is.
with tiles you can have some special tile=255 marker to mark unknown, and have the client send a request off to the server for all the tiles in some screen-sizedish block whenever you encounter it, then you'll not stall while waiting for the entire map right at the start. Maybe set all 255s to 254 in the block once a request has been sent for that block, and fill them in when they arrive.
The biggest issue with queries is what to draw while you're still waiting for the result of those queries. When teleporting, this can be a major issue.
If you dynamically change the tiles mid-game, the prespawn/signon buffers are static and thus not suited to this. Queries should still work, assuming all replies/updates are sent over the reliable channel, thus ensuring that they're all properly synced and ordered correctly. You'll have to notify the client about every tile which is changed, but you should not need to track what the client thinks the tiles are, if you're broadcasting all changes reliably.

[andrewj]

Many thanks Spike, that was really helpful.

P.S. I'm gonna try stuffing tile update messages in the reliable datagram, and increasing MAX_DATAGRAM (which is still 1024 in this antiquated codebase).

[Spike]

anything marked 'datagram' is unreliable. you don't want to exceed packet sizes of 1450 or you'll start having issues with broken routers, yes they still exist out there in the wild.
reliables have a max of 8192 or so in vanilla, which can generally be bumped up to 65535 without any other protocol changes.

[andrewj]

Yeah, at first I merely saw darkplaces and fitzquake had upped MAX_DATAGRAM (renamed in darkplaces) up to 32768 or 65535, but later I found the code in SV_SendClientDatagram which limits the size to 1400 for non-local clients.

[qbism]

Even local clients choke over wifi (at least on my particular network) so I commented out that exception in Fitzquake. Any multiplayer gets 1400

Code: Select all

	//johnfitz -- if client is nonlocal, use smaller max size so packets aren't fragmented
	//if (Q_strcmp (client->netconnection->address, "LOCAL") != 0)
        if (coop.value || deathmatch.value) //qbism- any multiplayer, even LOCAL will fail over wifi
		msg.maxsize = DATAGRAM_MTU;
	//johnfitz

[Baker]

Even local clients choke over wifi (at least on my particular network) so I commented out that exception in Fitzquake. Any multiplayer gets 1400

Code: Select all

	//johnfitz -- if client is nonlocal, use smaller max size so packets aren't fragmented
	//if (Q_strcmp (client->netconnection->address, "LOCAL") != 0)
        if (coop.value || deathmatch.value) //qbism- any multiplayer, even LOCAL will fail over wifi
		msg.maxsize = DATAGRAM_MTU;
	//johnfitz

"LOCAL" is the particular connection address for that client, btw. So if I am hosting a coop game, my connection address is "LOCAL" but the other player is something else.

And in my experience, the msg.maxsize in FitzQuake doesn't actually work at least for the signon buffer. Load up FitzQuake, do "maxplayers 2; sv_protocol 15; map start" and try to connect a GLQuake client. It will get a "Read error" because the datagram size exceeded the Quake default (which is 1024, so even if the #define for DATAGRAM_MTU wasn't 1400 which is still > 1024, it still doesn't actually work You can do #define DATAGRAM_MTU 1024 and it still fails to allow a GLQuake client to connect because signon datagrams are larger than 1024 ). You can do #define MAX_DATAGRAM 1024 and everything will work even with a GLQuake client trying to connect with Fitz using sv_protocol 15, Fitz has this as #define MAX_DATAGRAM 32000.

Additionally, if you really intended to detect a multiplayer game, the correct way is: if (svs.maxclients > 1)

Just sharing all the speedbumps I've encountered in the past, so damn much to learn ...

[qbism]

And in my experience, the msg.maxsize in FitzQuake doesn't actually work at least for the signon buffer. Load up FitzQuake, do "maxplayers 2; sv_protocol 15; map start" and try to connect a GLQuake client.

Additionally, if you really intended to detect a multiplayer game, the correct way is: if (svs.maxclients > 1)

Thanks, I'll switch to svs.maxclients.

Do any increased-limits NQ engines bother with preserving message length limits for true version 15 protocol? Or preserving Fitzquake protocol limits for that matter? DirectQ MAX_MSGLEN is higher than standard Fitzquake for example. Vanilla protocols are preserved only as required for demo playback or mod compatibility.

[Baker]

Do any increased-limits NQ engines bother with preserving message length limits for true version 15 protocol?

JoeQuake supports 8192 entities and GLQuake clients can still connect to it. This is just my opinion, but if something is going to supposedly run sv_protocol 15 and WinQuake/GLQuake/DOSQuake can't connect to it, it is broken.

I wouldn't let this hold back any of your ideas or thoughts, whatever is wrong with FitzQuake will eventually be discovered. If I install MSVC 2010 C++ Express, I could setup a watch to find out where this datagram size breakage is occuring (MSVC 2008 C++ Express doesn't support global watches, and MSVC6 somehow can't do this task for reasons I could explain 4-5 days ago but can't recall right now.)

The main reason it isn't a priority for me for find out why it doesn't actually work in FitzQuake is because no one is using GLQuake or WinQuake (so many things are busted in those clients on modern hardware) so the information is seeking the solution for a problem that doesn't really exist.

[Yet if I want to master network protocols --- and I intend to --- I'd be rather feeble if I couldn't solve this one. So at some point, I'll set out to determine why FitzQuake's intended backwards compatibility is busted. When that time comes, it'll be posted here. Yet, like stated above, no one is actually using GLQuake/WinQuake.]

[Spike]

many limits won't cause issues until they're exceeded. in which case extended clients will survice, while vanilla clients would normally be kicked by the server shutting down.

datagram sizes are fairly often exceeded when you have too many ents visible at once. which means that's not one to blindly extend when serving protocol 15, annoyingly enough. it would be common enough that it would make claiming to use protocol 15 redundant if the server supports something better.
for demo recording, it won't make a difference really.

[mh]

DirectQ gets around this by not even pretending to be suitable for use as a server. I've always taken the stance that it's a client and if you want a server there are other, better options available (running ProQuake -dedicated from another dir on your machine is one option I occasionally recommend). DirectQ itself doesn't have -dedicated anymore and I'm sometimes tempted to remove -listen, just to ram the point home: it's a client.

The philosophy here is, I guess, "do one thing and do it well" rather than trying to do everything.

[metlslime]

Even local clients choke over wifi (at least on my particular network) so I commented out that exception in Fitzquake. Any multiplayer gets 1400

Code: Select all

	//johnfitz -- if client is nonlocal, use smaller max size so packets aren't fragmented
	//if (Q_strcmp (client->netconnection->address, "LOCAL") != 0)
        if (coop.value || deathmatch.value) //qbism- any multiplayer, even LOCAL will fail over wifi
		msg.maxsize = DATAGRAM_MTU;
	//johnfitz

"LOCAL" means the same machine as the server ... LAN clients are not "LOCAL" in this sense, unless i misunderstand what you're saying.