Easier LAN coop play of classic Quake

[Baker]

Spike, thanks for your sv_stufftext trick here.

I've realize that if I setup an http server, it should not have a specific port and it should tell clients the port when they connect.

So I'll be reusing that already. I also will probably start using your extensions method when that time comes.

Then again, sadly, may it should have a specific port so a user could set their firewall options if hosting a non-local game. Not sure. NQ requires about all the ports to be open anyway for UDP. Hmmm.

Random port for now ... I can always refine it some later. [Losing some of the advantage of UDP transfer here ...]

[Baker]

if you want speed, the easiest way is to just make the server act as an http server too.

OR, you will be more pragmatic and set a low footprint HTTP server to handle the downloads in a much nicer, efficient way.
Just because you can throw the kitchen sink into the engine, it doesn't mean you should.

As it turns out, you were quite correct.

I can have it done now (estimate 2 hours from now or less) and perfect in the future ...

[Baker]

using fte's sys_win.c, you could create a download thread with this:

Code: Select all

struct dlthreadargs
{
    char *url;
    int done;
    void *threadhandle;
};
..

http://forums.inside3d.com/viewtopic.ph ... ant#p36627

Finally used the multi-threaded example you provided 2 years ago. Needed something to use it on, never could think of anything that "needed" it. Also somewhat has avoided multi-threading because of the variations required for multiplatform ...

[I really hope someday that in C ... well ... I wish Microsoft would adopt C11 in the future ... so there aren't 6 different ways to do the same time with #ifdefs depending on compiler/platform/yada ... But they won't ... and life goes on ...]

[Baker]

Note to self: Probably going to add some form of ...

1) Coop only .ent files. Like <mapname>.coop.ent

So some map that is a masterpiece with a single design flaw -- like a barrier that comes down, doesn't go back or something (frostbite, lunsp1, sm140 or whatever that one was --- can get "dealt" with.

Thinking: Maybe come up with a "smart" external entity file that only adds or removes entities from the entities string? Instead of repeating the entire thing?

Some ent files are quite hefty in single player.

The number of single player releases with a small design flaw in a coop is maybe 15%.

[Spike]

I still want a proper in-game .ent editor. one that's easy to use. give it a decent filter for game modes and you won't need coop-specific etc ent files.
one thing that bugs me about smc is that that mod has lots of different settings, with lots of different .ent files. the mod needs to control which ent files are loaded itself, on account of all the different settings which the engine can't really be expected to track.
so by all means guess the gamemode if there's none specified, but otherwise let the gamecode be specific.
tbh I'm more a fan of getting the gamecode to do it itself, though that does imply editing all mods to actually do that.

[Baker]

I still want a proper in-game .ent editor. one that's easy to use.

I have a different road to a similar place in mind. And I'm going about it in a different way.

I think a nice .ent editor means little if people are prevented from playing Quake mods. And they essentially are. If a mod isn't running on a server for someone to just easily connect, it is intimidating for someone. All the rules and complexities to get something running.

That's just my opinion. But I feel unless Quake's content is easily exposed, interest in heavy lifting won't be as high as it should be. [i.e. you have more interest in modding Quake if you get to try the fun stuff ... ]

about smc is that that mod has lots of different settings, with lots of different .ent files

I'm targetting exposing traditional Quake single player mods. The ones that don't deviate too much from the established expected QuakeC and don't have any features to play with it.

Historically all the mods that try to push the edges end up with arguably --- maybe not hacks -- but not forward thinking design, in part because I imagine it is a lot of fun to do it and if the flexibility is limited a bit due to something fun but with a price --- I get that. Too much of that and the mod becomes hard to maintain or understand or it begins to have trouble with its own weight ...

I'm looking at coop specific .ent files as a touchup without doing something terrible like messing with the .bsp. Mild stuff ...

[Spike]

that's kinda why I made addons (aka mutators) for fte, as a potential fix-all for adding stuff like ent types or whatever to mods that never had them, without having to hack the mods.
of course, such things generally need to have a nice enough ui, a noticable number of options, and sufficient mind-share for people to really care about it. Still, the engine part is done, just the ui+mods needed now, to which all I can really say is 'meh'.

yeah, its probably easier and simpler to use existing game mode convensions, and add coop and dmmX .ent prefixes or something. mods can always add extra ones.
I'm somewhat paranoid about distinguishing between coop and single player, kinda gives people an excuse to not bother. still, it can't make things worse, just make sure it tries the other as a fallback (and dmmX falling back on dm too, etc).

[Baker]

This has ended up being a more interesting mission where interesting = dealing with issues. Not logic issues.

I was considering running miniweb.exe (incredibly well designed little http server) via CreateProcess (server side) and handling the downloads via libcurl (client side). Stupid Windows firewall did a prompt about permissions for miniweb.exe somewhere along the way --- which I view as unacceptable --- so I'm DLLing it (it only has about 15 source files and is multiplaform).

As predicted, over wireless I get about 1 MB/sec download. I thought connecting an ethernet cable would massively increase that rate. Nope. Some sort of bottleneck (miniweb? curl? Some networking limitation about <x>, <y>, <z>? ) keeps it at 1 MB/sec even via ethernet cable.

So something like Travail (100 MB of stuff) is still not going to super-automatic-now-now-instant transfer over to another coop machine. It'll be 90 seconds or so.

[There is nothing about this that wouldn't work over the internet as-is, i.e. not LAN, but I just really don't care about that for too many different reasons ... ]

[Spike]

well, all I can really say is that you'll a dedicated thread on both ends in order to feed/serve the tcp connection. if the data backs up a lot, you'll fill your tcp window and stall, and that'll happen really easily if you have some video renderer limiting itself to 60 fps.
You can also increase the size of the tcp receive/transmit buffers. do that before connecting/accepting and you'll hopefully also increase the size of your tcp window too.
Or do both, to ensure there's really no limits there.

I thought you'd probably end up doing it in-process.

wifi should be able to achieve 54 mbps, so 4-5 MB per second isn't unreasonable (assuming its not a terrible connection). 100mbps ethernet should get 10 MB/s, and gigabit lan would be near instant if you're not waiting for disk access on both sides. Window sizes that large would be crazy though, hence the threads.

personally, my adsl internet connection's download speed maxes out at 700KB/s, but upload is much less. You'll find other people's connections can get much faster (grr, fiber). The rest depends on the server.

[Baker]

Stalled some ... thinking about a strategy to disallow stupidity:

1. Dumb gamedir names.
2. Someone using irregular files.
3. Sloppy mofo with contaminated files.
4. Guy with oddball files in id1

Meanwhile ... thinking about how games using -quoth don't actually use any files from -hipnotic. Just the HUD.

This is why with, say, ProQuake's http download the default archive contains commonly known maps, models, sounds (DM, CA, CTF, Slide, RQuake) and anything aside from those wasn't in the archive.

Like: Your work in-progress map that you keep doing new versions of with the same name, etc, etc.

[Spike]

yeah, in fte there's a policy where it tries to download paks-before-files, and apply versioning to the paks.
this ensures the client always has access to the right shaders etc, but also prevents one server's packages from infecting other servers (until the compatibility thing of downloading files not in pak thing, of course... bah. though in reality, such files could/should be versioned too).

More interestingly though, I've got my mind on websocket connections for playing right now, so I'm not far off just enabling an http server by default (quite tempted to include some html code to embed relevent fte plugins too...).

[Baker]

yeah, in fte there's a policy where it tries to download paks-before-files, and apply versioning to the paks.

That quickly was a conclusion I reached. Then I got to the "verification issue".

I'm going back to my first plan ... no installed mods. They sit around compressed, on gamedir switch, contents are extracted to a designated caches folder ( ... AppData\Roaming\<engine name>\caches ) ... and that will become a Quake path component.

[connection handshake] A connecting client will get the virtual gamedir source ("travail.zip md5:546a97f9bb04803a3e0191de065caed5") which the host can offer and then client does likewise.

[model/sound precache phase] At that point, the client will handle maps/models/sounds like normal.

Haven't decided how to handle the unknown mod issue. Likely answer = not going to bother? Or unknown mod gets sandboxed to AppData\Roaming\<engine name>\caches which gets deleted on engine startup and/or gamedir change. [Firewalled off untrusted stuff option, I suppose]

This would let known established mods get the "front door" treatment (and easily be safely shared between server to client) and keep "unknown" mods out of the Quake folder and get wiped, yet still played.

[Spike]

.zip?
eww.
pk3 at least has a well-defined root path. zips do not.
also, you'll need to redownload the entire zip on each revised pak file or whatever, or extra awkwardness.
side note: fte does the q3-compatible thing (what with its attempt at q3 protocol compat) and hashes only the file table rather than the entire pk3 - a noticable performance boost when you have 900mb mods.

[Baker]

.zip?
eww.
pk3 at least has a well-defined root path. zips do not.

I have to use .zip for historical purposes, otherwise I am modifying the archive that the mod was provided in (I would be messing with the author's archive).

Finding the rootpath is slightly annoying, but I have a method for that is not too hard.

Most of the coop and single player works were completed long ago, there are relatively few new ones. And they are sorely under exposed, in my opinion.

[Baker]

More interestingly though, I've got my mind on websocket connections for playing right now, so I'm not far off just enabling an http server by default (quite tempted to include some html code to embed relevent fte plugins too...).

Sounds interesting, a natural extension of the web browser plug-in version of FTE I assume?

side note: fte does the q3-compatible thing (what with its attempt at q3 protocol compat) and hashes only the file table rather than the entire pk3 - a noticable performance boost when you have 900mb mods.

That is a good trick. I wasn't aware Q3 did that.

Rhetorical question: How can you be certain the entire archive is intact though? Rhetorical because I bet adding the filesize table + the size of the files compressed sizes + <unknown overhead X> = archive size.