Fileplanet closing down, archival in progress

[qbism]

Any suggestions how to still make it public? Otherwise we could still save it but no-one could use it.

Expand contents to a web server. Use HTTracker or similar automatic clientside website backup tool to glean content that is exposed via links. Any files that are unlinked could be private. I have no idea if it would really work with this type of backup.

[Spirit]

Nope, to do that we would need to crawl the whole internet to see what files are publically linked. And we would need a time machine to do that for all days from when they started hosting files. And also BBSs, mailing lists and IRC etc. It is a approach from the wrong side.

We have access to the raw file tree via FTP.

For Fileplanet it is different, we crawled that from the website end so we know what was public and what not.

Actually the FP staff used a metric like that to determine what files to keep (if I understood it correctly), files that did not get any hits in a timespan they checked will/are/might be deleted.

Well, archive first, think later. Maybe we can find a way to make it public in a good way.

I will personally grab a copy of the planetquake stuff and publish a file list. With a proper request and reasoning (proof) I could send out files from that then.

[Spirit]

This is taking so long.

We should have all files at the end of this month.

Re the private files. You could always download them when you knew the URL. How about we upload them at archive.org without them being searchable in the index? That would be the exact same way how they were hosted previously. Item names at archive.org will be created in a easy and readable way so that you can easily go from "original download location at fileplanet" to "item at archive.org".

[Baker]

I just want to say that the archival efforts that you (and Chip and others) have done have been invaluable.

If all those FilePlanet downloads were permanently lost, that would be a rather large void considering how much was hosted there.

[Spirit]

I rented a bigger server (donate now!) so uploads are going live sooner now:

http://archive.org/search.php?query=sub ... eplanet%22

We got a "no" from the Internet Archive guys about uploading all the files seperately so you will get massive Tarballs instead. Copy the download link and append a / to browse them. I will provide a nice interface with direct links to single files some day.

[Spirit]

Getting really close to completion now for real. The 9.3 terabyte of public files (/ftp1/) are all uploaded to archive.org, they need some cleaning up though. The entry point will be https://archive.org/details/archiveteam-fileplanet (right now only some of the items are in there).

I do now have a copy of all the older files (/ftp2/) locally and they are also uploaded at archive.org (except a last one that is giving me trouble) but non-readable because of the afore-mentioned privacy issues.

I just verified the consistency of the ftp1 archives against a file list. Once I get the last ftp2 directory (stupid planetelderscrolls) fully downloaded, I will verify all of those too.

[Baker]

Thanks for all the work you have continually done (and LordHavoc too for QuakeDev and Chip's work too) to ensure that historical Quake mod source codes and other creations are not gone forever. There are tons of great and innovative things many different people did and ideas in those works. This game will likely be of interest even a 100 years from now just from historical reasons alone, but those aren't the reasons that interest me.

[Spirit]

We finished archiving a long time ago.

I am now finally working on a frontend. Very WIP but functional https://www.quaddicted.com/stuff/temp/f ... name=quake

Not fancy, but you can download fine.

If someone wants to collaborate, please shout. It's php & sqlite3. Great features to have would be a realtime search box and pagination. I have more metadata, screenshots etc, which I will join later. I will probably also properly restore the timestamps, noticed the "23:00:00" just now. I have them accurate to the minute.

[leileilol]

oh wow, you're an unsung hero on preserving video game patches as well.

and subscriber-exclusive betas! yay!

[Spirit]

Thanks!

Yeah, there is nice stuff inside. I should add that the Direct2Disk (or whatitscalled) things in /dd/ are encrypted installers so don't waste your time downloading those (and don't speak about them).

I meant to work on it today, btu didnt. So, some more info:

This does not include the ftp2 stuff, that's the "hosted files" from the planets etc.

You can grab the db at https://www.quaddicted.com/stuff/temp/f ... .sqlite.7z

Please do not mass-download via this interface. If you are after big chunks, please consider downloading the big TARs completely instead. I am not sure how IA's tarview system would handle many hits. I still have to ask them that...

If I knew how to do so nicely, I would add a browsing interface.

Rather don't spread this yet, it's not meant to be public to "the world".

[leileilol]

yeah I noticed that after it took some time to initiate a 100mb download of one file, I assume that's tar extraction on demand

[Spirit]

I added listing all files from a directory (and its subdirectories): https://www.quaddicted.com/stuff/temp/f ... ion/quake/