Fileplanet closing down, archival in progress

Discuss anything not covered by any of the other categories.
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Fileplanet closing down, archival in progress

Post by Spirit »

IGN is killing more gaming history: Fileplanet
Note: FilePlanet is no longer being updated and is in the process of being archived.
http://www.gamefront.com/breaking-ign-t ... fileplanet has some more info.

I am downloading like crazy but could really use some help. It seems not too urgent but you never know and the sooner we finish, the better.

See http://archiveteam.org/index.php?title=Fileplanet for how you can help (mostly with a Linux server, lots of space and good upstream bandwidth).

My goal is to provide a ftp-like structure to all the files in the end with the downloads being served by archive.org. The Fileplanet downloads are served from a very nice directory structure so this should not be too hard.

Random example:
action/half-life/levels/misc/cream.zip
action/half-life/models/@ss@ssin.zip
action/quake/official/q1source.zip
action/quake3/modifications/dm/decimal.zip
action/quake3/skins/Khazgod.zip
action/unreal/levels/dm/dmacanal.zip
bgchronicles/unoff/SD_Fast_Play.zip
fpnew/sports/deerhunter/dh2/maps/RM05.exe
frontends/fragfinder.zip
movies/enclave11921.mpg
movies/force_commander.zip
movies/neverwinter.mov.zip
patches/172ifr.zip
patches/Madden2000Patch4.zip
patches/NerfPatch.exe
patches/PharaohEnhancementPack.EXE
patches/TBass3DP2.exe
patches/U9patch107.exe
patches/WoTServerPatch.exe
patches/aow11.exe
patches/s3old156.exe
planetquake/cronos/tier4.zip
planetshogo/levels/OF_Maniac_Kickass.zip
planetstarsiege/tribes/misc/WKeyKill.zip
planetunreal/mutation/robinhood.zip
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
Baker
Posts: 3666
Joined: Tue Mar 14, 2006 5:15 am

Re: Fileplanet closing down, archival in progress

Post by Baker »

I don't know what we'd do without you and Chip.
The night is young. How else can I annoy the world before sunsrise? 8) Inquisitive minds want to know ! And if they don't -- well like that ever has stopped me before ..
Chip
Posts: 575
Joined: Wed Jan 21, 2009 9:12 am
Location: Dublin, Ireland
Contact:

Re: Fileplanet closing down, archival in progress

Post by Chip »

I'd like to help on this and archive the Quake related stuff, but I don't have that much space available.

I could use Amazon to host files as the costs are fairly small. How do I get the Quake files? Where is that nice directory structure? I don't use Linux.
QuakeWiki
getButterfly - WordPress Support Services
Roo Holidays

Fear not the dark, but what the dark hides.
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Fileplanet closing down, archival in progress

Post by revelator »

dont mind helping either though my once impressive disk space has gone awoll (curtesy of my many projects :S ) but i do have about 100 gig i could spare.
Productivity is a state of mind.
leileilol
Posts: 2783
Joined: Fri Oct 15, 2004 3:23 am

Re: Fileplanet closing down, archival in progress

Post by leileilol »

holy CRAP schbrid is doing a bang-on good job so far. Where was he when we needed him when happypuppy.com was sold!! :shock:

I wonder what'll happen when someone complains about being mirrored without consent though. There's one "K" name who would sue a site over screenshots. I fear the guy even for referring to the first letter.
i should not be here
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Re: Fileplanet closing down, archival in progress

Post by Spirit »

Cheers!

Chip: For Quake files it currently is not too interesting. The structure is exposed by their download locations, ie where they serve the actual download from. Newer files do not have that anymore. :( But we still get nice metadata down from the download pages and more.

reckless: Got a decent upstream (>1MB/s)? Would be best if you could join the IRC channel, #fireplanet on Efnet.

I estimate the full thing to be at ~4 Terabytes. So far we got almost 500GB down. The big files are still ahead. Worst case would be a total of 7TB.
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
leileilol
Posts: 2783
Joined: Fri Oct 15, 2004 3:23 am

Re: Fileplanet closing down, archival in progress

Post by leileilol »

Any way to get date ranges from the file ranges? I'm interested in '99/'00 junk
i should not be here
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Re: Fileplanet closing down, archival in progress

Post by Spirit »

The IDs seem to be incrementing over time, so low ID = older file. I am making sure that we store the log files in each item, see the files_*.log
Be aware that you can browse the tar files directly, copy their direct download link and add a / .
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Re: Fileplanet closing down, archival in progress

Post by Spirit »

We got ~6 Terabytes down now and that is pretty much what we could easily get. Plans for a simple websites interface once FP is gone are well underway.

If someone knows someone working near Fileplanet, PLEASE get me a contact.
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Re: Fileplanet closing down, archival in progress

Post by Spirit »

All the gamespy forums are gone, planetquake forums, planetdoom forums, pretty much everything. Allegedly 40 million posts (a lot of actual spam though). I got about 20 million archived.
I'll try to get them up at http://gamespy-archives.quaddicted.com/ but I guess I do not have enough space for the extracted stuff. Anyways, you can find compressed archives there. I will upload them to archive.org too of course.
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
revelator
Posts: 2621
Joined: Thu Jan 24, 2008 12:04 pm
Location: inside tha debugger

Re: Fileplanet closing down, archival in progress

Post by revelator »

upstream approx 2mb but it seems you got it sorted. Sorry for not chipping in earlier sometimes the forum doesnt show new post in threads im watching (?).
Productivity is a state of mind.
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Re: Fileplanet closing down, archival in progress

Post by Spirit »

Heh, I actually got a free server with excellent connectivity to both fileplanet and archive.org so I can often fully max out its 100mbit/s. Fileplanet is 99% done. Anything below that is considered slowdown now. ;)

Next come the files that are "hidden", I'll grep the forums and maybe some planetnetwork site mirrors for URLs.
This is fun!
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Re: Fileplanet closing down, archival in progress

Post by Spirit »

Things are slow but progress is fantastic. We found some supporters at IGN and now have ftp access to all fileplanet and all of the hosted files (from the hosted sites) aka the holy grail. It will be around 11TB in the end.

Sneak peek of the "hosted files" directory: http://www.quaddicted.com/stuff/temp/ftp2snippet.png


Oh, and http://gamespy-archives.quaddicted.com/forums/ has the forums I got. No nice frontpage, if someone wants to help with that, it would be most welcome. No idea if I can keep hosting it like that, it's actually a .tar.bz2 that I mounted with avfs which eats a lot of cpu 24/7.
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
leileilol
Posts: 2783
Joined: Fri Oct 15, 2004 3:23 am

Re: Fileplanet closing down, archival in progress

Post by leileilol »

An important thing to note - there are cases when some used hosted files for sending private files (pre-rapidshare age after all). Keep in mind that there' may be stuff not deemed for public consumption, like say, private sourcecode, someone's work files for a model, or something along those lines. I know a few people (who were hosted by gs) that sent private files through the hosted file system because email attachments for even 5mb files would be cumbersomely inconvenient.

Mass file archivery bears a huge responsibility. There's a fine line between preserving computer culture and violating other's personal rights, and there's a bit of crossing with it when you go by doing it this way. Would you archive a failing cloud storage service for "fuck your morals, this is internet history"? This is no different, even though the majority of the files have been made public for years, and IGN's owners may not even be aware of the privacy issues.

11TB is huge! Any /qca/ remnants?
i should not be here
Spirit
Posts: 1065
Joined: Sat Nov 20, 2004 9:00 pm
Contact:

Re: Fileplanet closing down, archival in progress

Post by Spirit »

Aw damn, I wish you had not said that.
Any suggestions how to still make it public? Otherwise we could still save it but no-one could use it.

The Fileplanet stuff seems not affected though, that is the majority (9.5TB).

No /qca/ at all (tQER_qca0.zip though).
Improve Quaddicted, send me a pull request: https://github.com/SpiritQuaddicted/Quaddicted-reviews
Post Reply