Sockets programming

[Baker]

Sockets programming is funny.

It is both ridiculously easy and ridiculously hard at the same time.

You can get something up and running in no time flat, but if you want to dot all the i's and cross all the t's ---- multi-platform, ip4 + ip6 support, tcp + udp and validate everything, it can be incredibly time consuming. There are many asterisks to deal with.

[Spike]

its quite simple... ditch ipv4, stick to only one of udp or tcp, write for bsd sockets and add some extra #defines to cope with winsock dodgyness. easy. ish.

but yeah, validating everything to avoid exploits... good luck with that.

[Baker]

Yeah, I'm pretty far down that road. In NetQuake, there is no reason net_wins.c and net_udp.c should be different files -- they aren't different files even in Quakeworld in the 1999 source release and the differences between bsd sockets and winsock are fairly light (close vs. closesocket, ioctl vs. ioctlsocket, etc.) . I've also discovered on Windows strerror sucks at converting a Winsock error code # to a string.

Actually I'm using both TCP and UDP, similar to ezQuake. Mostly because I want to accumulate the experience of working with both because many situations TCP probably a better solution (file transfer and the like).

The plug-in architecture I see in FTE is intriguing. I'm not going to think about that too much at the moment but I find it very interesting how that works.

I haven't checked how FTE pings servers, but I have read about SOCK_RAW not working on modern operating systems for classic ping operations and how on Windows IcmpSendEcho is how gets done.

The documentation on IPv6 sucks a bit. But I'm working through that. Mostly use AF_INET vs. AF_INET6 and a few functions.

[Baker]

I didn't realize how old ip6 was:

Code: Select all

/* Copyright (c) 1996 by Internet Software Consortium.

Code: Select all

/*
 * WARNING: Don't even consider trying to compile this on a system where
 * sizeof(int) < 4.  sizeof(int) > 4 is fine; all the world's not a VAX.
 */

^^ a little bit of comedy. VAX sounded familiar but I wasn't 100% what it was: http://en.wikipedia.org/wiki/VAX

Code: Select all

/* const char *
 * inet_ntop6(src, dst, size)
 *	convert IPv6 binary address into presentation (printable) format
 * author:
 *	Paul Vixie, 1996.
 */
static const char *
inet_ntop6(src, dst, size)
	const u_char *src;
	char *dst;
	size_t size;
{

[Spike]

vista, linux, *bsd support hybrid sockets (which typically have to be explicitly enabled).
xp is dual stack (ie: not hybrid dual stack). 2k requires extra drivers. both will report an error if you try enabling hybrid mode of an ipv6 socket, or outright fail to create an ipv6 socket.

a hybrid socket will report ipv6 addresses as ::ffff:XXXX:XXXX (typically written as ::ffff:X.X.X.X instead). once you have a hybrid socket, you can treat the peer as if they're an ipv6 client even if they're ipv4, or if you only have an ipv4 address.

if you care about xp, you can either just create two sockets, or use ipv4 instead of ipv6. the size of your addresses differ, but normally you shouldn't care about anything else.
fte cares because it does IP filtering and does some weird crap to try to ignore the peer's port number for old routers, but those routers were seriously dodgy and are almost guarenteed to have been replaced by now. the rest is mostlty just string->address and address->string. the rest of the code really shouldn't need to care, so long as any memcmps have the correct address size (or you pad them with 0s, or use a netadr_t abstraction or something).

fte's plug-in archetecture is a bit of fun more than anything else. it does gracefullyish handle tcp vs udp, but channeling quake's packets over irc is folly except for comic effect. note that NQ has a plug-in archetecture too - that's what nq's net_win is all about, allowing it to do both ip and ipx. nq worked over null modem cables and other stuff like that back in the day.

unlike what you say, vanilla QW _DOES_ have separate net_wins.c for windows and net_udp.c for unix. I personally favour using #defines to cover the differences, but maybe that's just me. just try to avoid lots of random #ifdefs because those make things unreadable. take care with later versions of windows sdks defining both EWOULDBLOCK and WSAEWOULDBLOCK (and friends) with different values, winsock uses ONLY the latter while unix doesn't have this madness.

sock_raw needs admin access, its never really been usable in linux as a result. its only windows that basically defaulted to everyone running as an admin anyway, which is why it was _generally_ allowed there.

[Baker]

vista, linux, *bsd support hybrid sockets (which typically have to be explicitly enabled).
xp is dual stack (ie: not hybrid dual stack). 2k requires extra drivers. both will report an error if you try enabling hybrid mode of an ipv6 socket, or outright fail to create an ipv6 socket.

a hybrid socket will report ipv6 addresses as ::ffff:XXXX:XXXX (typically written as ::ffff:X.X.X.X instead). once you have a hybrid socket, you can treat the peer as if they're an ipv6 client even if they're ipv4, or if you only have an ipv4 address.

This is what I've been trying to grapple with and research the last 24 hours, trying make sure I sure I follow the guidelines in this http://www.kame.net/newsletter/19980604/ along with other things I've picked up. i.e., make sure everything uses sockaddr_storage, etc.

if you care about xp, you can either just create two sockets, or use ipv4 instead of ipv6. the size of your addresses differ, but normally you shouldn't care about anything else.

I am aware of that on XP, but haven't crossed that bridge yet. I'll have 2 sockets open for UDP vs. TCP, I will end up trying to the IP4 vs. IP6 socket method just to examine it.

take care with later versions of windows sdks defining both EWOULDBLOCK and WSAEWOULDBLOCK (and friends) with different values, winsock uses ONLY the latter while unix doesn't have this madness.

Egads! Did not know. Thanks.

[Baker]

On ipv4, a server might be represented as 66.55.44.33:26000 for port 26000.

Colons are out for ipv6, so how should that be displayed in ipv6?

Update: https://[2001:db8:85a3:8d38a2e:370:7348]:443/ <---- well that looks nasty, but then ipv6 addresses aren't friendly to start off with

Or something like this [::ffff:452f:a2cb]:26000

[Spike]

if its more than just an address (ie: in a url where a port might be specified), you need the [] around it, yes.
note that the [] should presumably be possible around simple hostnames too.
noone is really going to want to enter full ipv6 addresses anyway. and if they are, they'll probably be simple ones like ::1 for localhost.

[Baker]

Well, would be quite hard to connect to a server if the connect command doesn't support it.

But yeah, I get your point. (And nothng would prevent old fashioned use of the "port" command i.e. "port 26000; connect <address>")

[Spike]

the connect function only accepts a sockaddr_xxx, not a string. the port is typically implied, so you can just enter a hostname and be done. typing full literals is crazy talk that only weird people do. and even then they probably copy+paste it anyway.

[Baker]

It is rather disappointing on Windows that I keep hitting "incompatibilities" with the standard.

Some examples: Windows has no inet_pton or inet_ntop built-in and getaddrinfo run against gethostname's return value doesn't do what the deprecated ipv4 gethostbyname function does.

For instance, getaddrinfo is returning 127.0.0.1 or ::1 instead of 192.168.1.x

Windows has their own non-standard functions Inet_pton and Inet_ntop avaible on Windows 8 :roll: I ended up snagging inet_pton equivalents.

I guess I don't fully understand why a company with so many resources can screw up stuff like this. I guess I will need to use deprecated gethostbyname on Windows, which is maddeningly stupid.

I'm not sure what to do for IPv6 since addresses the address have more information packed into them, you can't just convert an ipv4 address to ipv6 and have the right result (i.e. fe80::/10).

[Spike]

getaddrinfo("localhost",...) should of course return 127.0.0.1 and ::1 depending on the address family.
logically gethostbyname should NOT intrinsically know the local name, but should use the same routing tables as those available to other machines (ie: dns). this allows you to be sure that using the same name on multiple hosts on the same lan will actually get the same result. this is a good thing (and it has always been this way on the unixes), but leaves a gap when you want the same thing elsewhere. the fact of the matter is that if you're actually trying to report your ip address to other machines then you have already lost (NAT issues). it really depends how strict you want to be, but looking up the local name has always been a windows extension/quirk.
for windows, fte does use gethostname+getaddrinfo to report a list of local addresses for any socket with an 'any' address binding on windows, so I don't know what the problem you're having is. as said above this understandably does not work in unix.
for linux/glibc systems, fte uses getifaddrs. for other systems it just reports 0.0.0.0 or so.
why do you even need inet_pton when you have getaddrinfo / getnameinfo?

[Baker]

getaddrinfo("localhost",...) should of course return 127.0.0.1 and ::1 depending on the address family.
logically gethostbyname should NOT intrinsically know the local name, but should use the same routing tables as those available to other machines (ie: dns). this allows you to be sure that using the same name on multiple hosts on the same lan will actually get the same result. this is a good thing (and it has always been this way on the unixes)

I came to this conclusion before I read this post and decided the right way to solve the "LAN" issue is broadcast / multicast.

My LAN ip address of 192.168.1.x is meaningless outside the LAN and I could even have multiple local ips.

Especially if I am bound to INADDR_ANY or the IPv6 equivalent (0's either way), there isn't necessarily a single right answer.

[Baker]

why do you even need inet_pton when you have getaddrinfo / getnameinfo?

I am using it to read sockaddr_storage structures to verify their contents.

[Baker]

Prototype construction (on Windows using highly transportable coding practices, coding like I'm using BSD sockets on Unix using Beej (site) as a reference):

Automatic IP4 client to IP4 server working.
Automatic IP6 client to IP6 server working.
Automatic IP4 client to IP6 server working. So far. I've tried to construct a breaking IP4 client to IP6 server scenario and haven't achieved one yet that breaks the IP4 to IP6.

Which bothers me.

I wrote the code in ip-address agnostic style, but didn't implement anything special on the server side except setsockopt IPV6_V6ONLY to off. Which according to what I understand shouldn't be enough by itself. Which leads me to believe it is being done by the operating system (Windows). We'll see. I haven't run it through enough of gauntlet yet. I'm expecting some scenario to break it, I just need to find it.

The fun stuff:

1) sockaddr_storage every where
2) no use of AF_INET vs. AF_INET6 except in (struct sockaddr_storage *)blah->ss_family and let everything use that.
3) Because of #2, have to evaluate addresses really early so all the socket setup knows what to do.
4) No use of deprecated functions or ancient ways of doing things.

Plenty of more work to do ... I'm thinking about attempting to kill off my use of libcurl after I finish this.