Synchronous Download Function (Win32)

[Baker]

I wrote this mostly because having libcurl and zlib1 in a folder pisses me off. Especially since there are different versions of zlib1.

This is native Win32, so no DLL is required.

Don't get me wrong, I would prefer the libcurl solution if somehow both libcurl and zlib1 could be compiled into the binary without doing something crazy.

This function could easily be modified to replace the Curl-based download solutions (just feed it a callback function like Host_Frame or what not and do that instead of the update messages --- then it isn't synchronous anymore ). I hate seeing engines break because some goofy DLL in the Quake folder isn't the right version.

Example: Sys_Download ("http://www.quaddicted.com/filebase/warpspasm.zip", "warpspasm.zip")

Code: Select all

#include <wininet.h>
#pragma comment(lib, "wininet.lib")
qboolean Sys_Download (const char *strSourceUrl, const char *qdestpath)
{

	HINTERNET	hINet = NULL;
	HINTERNET	hFile = NULL;
	FILE			*fp;
	int			filesize;
	char			outfile[MAX_OSPATH];
	qboolean		success = false;

	sprintf (outfile, "%s/%s", com_gamedir, qdestpath);

	do
	{
		// Handle all the "fail" situations

		hINet = InternetOpen("Quake", INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0);
		if (hINet == NULL) 
		{
			Con_Printf ("Download failed; no internet\n");
			goto download_shutdown;
		}

		hFile = InternetOpenUrl(hINet, strSourceUrl, NULL, 0, INTERNET_FLAG_RELOAD, 0);
		if (hFile == NULL) 
		{
			Con_Printf ("Download failed: Unable to open URL: %s\n", strSourceUrl);
			goto download_shutdown;
		}

		// Get size
		{
			char buffer[64];
			DWORD length = sizeof(buffer);
			HttpQueryInfo (hFile, HTTP_QUERY_CONTENT_LENGTH, (LPVOID)&buffer, &length, NULL);

			filesize = Q_atoi (buffer);
		}


		// Load information to file. 
		fp = fopen(outfile, "wb");
		if (fp == NULL)
		{
			Con_Printf ("Download failed: unable to open file for writing '%s'\n", qdestpath);
			goto download_shutdown;
		}

	} while (0);

	// Read file.
	do
	{
		char	buffer[1024];
		DWORD	dwRead;
		int		cumulativeRead = 0;

		while (InternetReadFile(hFile, buffer, 1024, &dwRead))
		{
			// Every 256 KB, do an update message ....
			if ( ((cumulativeRead + dwRead) /262144) > (cumulativeRead / 262144) )
			{
				Con_Printf ("Progress: %i of %i (%3.2f %%) ...\n", cumulativeRead, filesize, ((cumulativeRead / (float)filesize) * 100.0f) );
				SCR_UpdateScreen ();
			}

			if (dwRead == 0)
				break;

			cumulativeRead += dwRead;

			fwrite(buffer, dwRead, 1, fp);
		}

		fclose(fp); 

		success = true;
	} while (0);


download_shutdown:

    if (hFile)	InternetCloseHandle(hFile);
    if (hINet)	InternetCloseHandle(hINet);

    return success;
}

[Spike]

Eww.

1: windows specific.
2: blocking, windows/input messages get nuked for the duration. TCP has a 2-hour timeout period.
3: gotos... and do{}while(0); with no breaks or continues or anything.
4: dependant upon internet exploder settings, external to your program (especially the 'enable cookies: prompt' option).
5: cookies+self-signed certs etc. Automatic prompt tries to appear but fails due to no window messages?


Yay.

1: https/ftp support

[Baker]

4: dependant upon internet exploder settings, external to your program (especially the 'enable cookies: prompt' option).
5: cookies+self-signed certs etc. Automatic prompt tries to appear but fails due to no window messages?

I never thought of that. I haven't used Internet Explorer since FireFox came out so it wasn't even on the radar in my head

On a Mac, it is perfectly normal to use to use native operating system call for downloading a file. But I wasn't thinking about how the above has a "IE tie-in".

I recognized the above solution as slightly sub-par, but I didn't think it was THAT bad (until you pointed the above out).

Crap.

Almost opted for an FTP oriented solution, but the shortness and the ease of the above made me think "Errr? What was the reason InternetOpen is crappy?"

/The "Windows only" thing seriously doesn't bother me, especially if other operating system equivalent code is commonly known.

Back to libcurl or maybe writing up some FTP API stuff. I really hate extra .dll file conflicts (zlib1, fmod, etc.).

[Spike]

there's an argument to disable the cookie ui, so that should help mitigate that.

popen+wget = yay... assuming you have wget installed.


with fte, I have some http+ftp download stuff that I wrote a while ago, using system sockets. I'm not going to claim that its perfect, but it works suffienctly for me. the biggest annoyance for me is that it lacks https support, but that's a general socket-layer annoyance and applies to certain other things like websockets+tcpconnect etc.
http itself is not an overly complicated protocol, but there's a lot of twiddles like redirects. any html resources with meta redirects are impossible to follow from a purely http aproach.

The windows-only thing isn't a major problem as you can always just #ifdef _WIN32 it anyway, with a fallback stub that just returns some sort of error. However, if you do intend to be portable, alternative implementations on different platforms are a pain. Its easier to test if its the same on all platforms.

You could always statically link. FTE statically links against zlib+libpng+libjpeg on windows, so no conflicts there.
Alternatively, app-controlled dynamic linking can be used as a way to query versions or to ensure that the whole thing doesn't blow up if someone shoved the wrong dll version in there.
Tbh, the biggest pain with dlls is with 32bit vs 64bit builds.
Statically linking will generally allow all the unused entrypoints of a library to be stripped out, giving smaller zip sizes etc, so there's more upsides to doing so than just the obvious.

[mh]

The Windows-only thing isn't that bothersome. Normally you'd write a wrapper interface around it that calls into whatever the appropriate API might be depending on your platform. This code can be seen as what the Windows version of that interface might look like.

The big problem is: where does the download mechanism get it's connection settings from? If you've a direct connection then that's cool, you just connect directly and it works. If you're behind a proxy or using dial-up (and remember that dial-up is still used by a lot of mobile broadband providers) then like it or not the connection settings have to be obtained from your browser (at least on Windows, no idea about other platforms) - because that's just the way things work (snarky comments about whether or not it was a good way to make things work won't change it).

[Spike]

if windows' 'Internet Properties' control panel tool didn't jumble up internet access and IE browser settings, then the settings in there might actually be more useful for browsers sane people might choose to use. Sadly that stuff is all ie-specific rather than generally useful.
But yes, configuring http proxies in every single app that might communicate with the internet is a real pain.
Which I suppose is the real advantage of an http proxy over a NAT...

[taniwha]

For avoiding dll issues: don't use dlls. Link statically. I do for QF (I cross-compile win32 bins using mingw in linux). All of QF's external deps (except opengl.dll) are statically linked.

[Baker]

For avoiding dll issues: don't use dlls. Link statically. I do for QF (I cross-compile win32 bins using mingw in linux). All of QF's external deps (except opengl.dll) are statically linked.

The MSDN docs on this subject aren't real clear and I'm not sure how to do it in Visual Studio 6 (nor 2008). I would think I'd need to specify the .dll location somewhere and do a flag somewhere (Code generation?).

[taniwha]

For static linking, normally one would skip the .dll and use a .lib. Since curl and zlib are both buildable from source anyway, you should be able to build .libs for them.

Absolute worst case (and this might be the craziness you were trying to avoid): include the sources in your build tree and project.

[mh]

For a zlib replacement, Q3A has all the necessary code and you can just copy and paste.

stb_image is a nice option for many image formats, but it only does 8-bpp png last time I checked.

[taniwha]

Gotta love that about the quake engines being GPLed: code can migrate back and forth between them as necessary