QuakeDev.com downtime and request for archival

[Chip]

Hey LH, when is quakedev.com expiring? Are you aware of the exact date? Are you aware if anyone wants to take it back?

I do not know the exact date.

As far as I know no one has offered the next monthly bill to keep the server running.

apolluwn has offered to take over (by migrating the site to a new host and maintaining it).

icculus has offered to archive the entire site in place, but that would not be maintained, simply a frozen archive.

It is my understanding that icculus has made a full backup of the server filesystem (not the user-visible front side but the entire server) a couple weeks ago.

I have a backup of the entire web and svn portions.

I do not know enough about good methods to back up the rest to pursue it.

If you can get me instructions on what files I should archive from the filesystem side, I'll happily grab the forums.

If we are to work out a new maintainership, it had better happen soon.

I can pay the bill a second time if necessary, but I need to be convinced people are going to act on it.

Well, thanks.

I wanted to take over the forums, maybe even the site for hosting, but without the costly SVN features. I'm a shared hosting reseller and I could afford hosting it, the domain name renewal is 12USD, isn't it? That's the yearly cost of any .com domain.

I don't need the filesystem, I need the SQL database, where all posts and threads and users are stored.

I was thinking of letting the domain expire and purchase it myself (if anyone else wants to do this, let me know). You could make an SQL dump by using phpMyAdmin. If the forums folder is backed up, that's even better. Post attachments are there.

I don't want a frozen archive, that would be no good, it wouldn't be dynamic as in searchable.

[apolluwn]

I do not know enough about good methods to back up the rest to pursue it.

There are many ways to accomplish this from the command-line if you do not have access to a GUI like phpmyadmin/webmin/etc to backup/dump the tables.

Remotely:
mysqldump --all-databases -hserver.com -uroot -p'passwd' | bzip2 > databases.sql.bz2

To accomplish remote connections from the root user, you may need to modify the host permissions for the root user.

This would require that the host permissions of the MySQL superuser are set to ANY(%) or, at the very least, the IP address that the connection to backup the database originates from.

mysql -uroot -h'passwd'

mysql> use mysql;
mysql> select host, user from user;

At this point it will output the user host permissions. Find the root user and if it is not set to % then remote connections will not be allowed for this user.

It should look something like this:

--cut--
| localhost | root |
--cut--

Still from the mysql command-line update it with:

mysql> update user set host='%' where user='root' and host='localhost';
mysql> flush privileges;

Output should be similar to this:

Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

Verify it updated correctly by running this command again:

mysql> select host, user from user;

Output for root user should now be:

--cut--
| % | root |
--cut--

or the IP address if it was done that way.

You can also accomplish this locally with the same basic command line omitting the hostname:

The user should default as root (you should be able to omit the -u switch), but if it doesn't you need only add the -u switch:

mysqldump --all-databases -uroot -p'passwd' | bzip2 > databases.sql.bz2

If you are only after a single database, or only a few of the databases you can use:

single database =

mysqldump -uroot -p'passwd' databasename | bzip2 > databasename.sql.bz2

or

several databases =

mysqldump -uroot -p'passwd' --databases databasename1 databasename2 etc etc | bzip2 > databases.sql.bz2

It is important to note that if you elect to use either the -A (--all-databases) or the -B (--databases) switch then it will dump all of those databases into a single sql file. Unless this is the behavior that you want you should probably stick to the backing up a single database at a time.

It is also not necessary to use bzip2 compression. You can change the commands to

mysql -uroot -p'passwd' [other options] > outputdatabase.sql

to output just the sql without compression. Or, you can substitute bzip2 with your program of choice. By default, the output goes to STDOUT so "> filename.sql" is necessary unless the database is very small.

If a full database backup is made then you should be aware that these can potentially be -VERY- large.

[Chip]

Now that's detailed. But, as LH is not familiar with servers, I'm sure someone with server access could do it, or someone with FTP access could upload a copy of phpMyAdmin and dump an SQL backup.

Indeed, a database backup could be big, but in this case, I'm sure it's not bigger than 64MB.

Anyway, this should be done pretty quick.

[Spirit]

Whatever is done, please keep the privacy and security implications of a full database dump in mind. There will be private messages inside. Passwords, hashes, mail address, names and so on.

[LordHavoc]

Whatever is done, please keep the privacy and security implications of a full database dump in mind. There will be private messages inside. Passwords, hashes, mail address, names and so on.

This is the primary reason I keep the root login for archival purposes closely guarded. This stuff is all private until someone proves they have a legitimate claim to their part of it.

I have rsynced the entire server (68GB, a 50GB rsync transfer as it turns out), icculus has a complete backup as well. Mine lacks user ownership but I only intend it as a resource to pull archives from.

I still need to do the mysql dumps or have Echon do that.

Speaking of Echon, the server has auto-renewed on his account again, so it's still going to be up into early June, but we still need to figure out what to do going forward, Echon would very much appreciate it if he didn't have to keep paying the bill.

[raptore]

I'm not finished reading the thread yet, but if this is still a problem, I am interested in helping to fund the bill for the site that is facing oblivion.

[Baker]

Anyone have a current URL for any mirror of the Quake Expo sites?

[LordHavoc]

Anyone have a current URL for any mirror of the Quake Expo sites?

They are not public in raw filesystem form, individual site owners can contact me for a backup of their site to put it back online, I may also service some requests for individual files (like released mod zips and such) to anyone wishing to make a mirror of the meaningful parts of the site.

Privacy must be respected so I don't have a public mirror of the raw folders myself and I review any requests carefully.

Unfortunately a number of the sites are dependent on mysql or other services that need to be copied and set up properly for a mirror to work properly, I can also supply these files (in both mysql dump form and also just the raw files).

[Baker]

I'm sure those would be quite a bit of work to try to get back online and a major time consumer . Annoyingly, it appears that Quake Expo 2001, 2005, 2006 are not in archive.org.

http://qexpo.quakedev.com/ ... http://wayback.archive.org/web/*/http:/ ... kedev.com/
http://qexpo2005.quakedev.com/ ... http://web.archive.org/web/*/http://qex ... kedev.com/
http://qexpo2001.quakedev.com/ ... http://wayback.archive.org/web/*/http:/ ... kedev.com/

The 2003 Quake Expo site is in archive.org: http://web.archive.org/web/200310090619 ... qexpo.com/
And obviously the 2008 and 2010 sites exist: http://qexpo.tastyspleen.net/ http://expo2011.quakewiki.net/

[Spirit]

They are archived at archive.org but because the domains currently serve no permitting robots.txt archive.org will not display them. I am uploading my 2001 and 2005 mirrors, not sure where/if I got an qexpo. one.

[Spirit]

Crap, my mirrors were done incompetently and would require some work to make them usable. http://www.quaddicted.com/webarchive/qe ... kedev.com/
Should not be too hard but I have other interests. Can provide tarballs if someone wants them. Those were "wget -m"ed I guess, unfortunately without better options.

[leileilol]

Didn't FrikaC dvdburn every qexpo?

[Baker]

FireFox switched my default search engine to Bing and I accidentally used Bing for a search.

It had the 2006 Quake Expo front page (without CSS) in the cache.

I tried to see if other Quake Expo pages might be cached by Bing, but I quickly discovered it really isn't very good for searching.

[Sure Chrome is a better browser than FireFox, but I don't really want megacorps running the whole internet ...]

[noobish]

If this is still something everyone would like back online, I could host it (with svn as well). I have a small hosting business and I would be willing to run it for donations only. Looks like a squatter might be on the domain name (it's up in January), so might have to pick a diff one.

I would need to know how big everyone's archives (uncompressed) are for planning purposes.

[Spirit]

And who are you to be trusted?