Author Topic: The Vault Preservation Project  (Read 6060 times)

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
The Vault Preservation Project
« on: September 28, 2012, 03:55:33 pm »


                <looking down...>

I'm worried. <and shaken>
You got that right, pretty black bird <gee, thanks boss>
Don't get a swelled head. Compliments merely show how worried... <and shaken>
...I am. Yup.

You guys know how optimistic and positive I try to be. <and the ladies know how...>
Hush, bird. No time for banter today. <grump>

Some of you may know I submitted my first stories to the Neverwinter Nights Vault in 1999 under Valen and that I've recently been trying to do what I can to revitalize it under Maximus. Now I'm worried (and shaken, yes, we know Bother). The Vault is a simply huge repository of more than a decade of community effort in diversifying and enhancing NwN and NwN2. It's in serious trouble. It is currently working in "read-only" mode.

What I am going to do about it:
I have a domain for the next 5 years with unlimited storage/bandwidth. I will be adding a "Vault Project" content type to it to allow the archiving of content salvaged from the Vault. While the Vault is viable it is meant to be a mirror, a back up of the Vault. Should something happen *shudder*, I hope it will act like (but do a better job at saving files) the Wayback Machine.

In fact, the VPP is going to push *all* my other projects to the back burner. (I'll still work on the CCC off-line Edit: and the Q window lights :-) I like that project :-).
Once I get the neverwintervault.net site set up to recieve Vault projects, I'm going to start salvaging the haks, as that is my personal highest priority - the custom content.

The procedure:
  • Go to the project list for the category I'm committing to. pull up the next project in the browser.
  • Create a sub-directory with the name of the project. save the webpage in that sub-directory to capture the project info (and comments - there's a lot of info in the comments).
  • Download all associated files (but not links to other projects) to that directory
  • Repeat. (The object here is to get each project *and it's info* into local storage ASAP)
  • When time permits, upload the projects to nwn-cc.
What I would like (very much!) is some help. Are there any people out there who have good bandwidth and large harddrives and time on their hands? Post here if you'll commit to a project category (or even a partial category like mods pages 1-5). Then go register on neverwintervault.net  so you can upload when I get the content type defined (this weekend).

Categories:
Edit: And *please* don't burnout on this project! I'm only asking for an hour or two a night, when you have time...

Caveat: this is an independent effort unaffiliated with the Vault, Maximus or IGN, with the exception that it is *not* meant to supplant or compete in any way with the Vault. It is meant to safeguard the community content.

I am an Admin for the Vault and I have no intention of abusing that trust. I am, in fact, seeking to coordinate this with my boss :-)

It is not meant to compete with the Nexus or the Moddb or any of the other excellent community sites in my sig. They wouldn't be in my sig if I didn't respect and, in some cases, love those sites.

<...to try and avoid stepping on toes>
               
               

               


                     Modifié par Rolo Kipp, 14 octobre 2012 - 07:17 .
                     
                  


            

Legacy_Michael DarkAngel

  • Hero Member
  • *****
  • Posts: 627
  • Karma: +0/-0
The Vault Preservation Project
« Reply #1 on: September 28, 2012, 04:44:24 pm »


               I'll offer what help I can.  My biggest issue is time.  I'll only have a few hours a night, a little more on weekends, but I have the bandwidth and HD space covered.

Might as well make my first commitment right at the top of the list -- NWN Screenshots.  Gotta start somewhere right?

'Posted
  MDA
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
The Vault Preservation Project
« Reply #2 on: September 28, 2012, 04:45:54 pm »


               <dances a little...>

WooHOO! *Bless* you, you dark angel of mercy ;-)

<...jig>
               
               

               
            

Legacy_virusman

  • Sr. Member
  • ****
  • Posts: 448
  • Karma: +0/-0
The Vault Preservation Project
« Reply #3 on: September 28, 2012, 05:21:00 pm »


               It may be much more efficient to write a web crawler (or use an existing one) that'd download all Vault content.
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
The Vault Preservation Project
« Reply #4 on: September 28, 2012, 05:27:38 pm »


               <not...>

I tried archiving the site with wget and there is simply so very many links on each page and sooooo much other content that it took 8 hours to get a very few pages. Someone better with WGet (or something) might be able to do better.
Edit: I do have access to the recent files over 25mb, but not to the project info the authors submitted or to the comments.

If someone could web-crawl the project pages on the vault and store all the entries in CSV or excel format, I can import them into the VPP content types (which is a drupal front-end on a mySQL db). Otherwise, I just figured to do it the old fashioned way, since many hands could make relatively short work of it.

I'm just hoping to have a relatively safe backup of the content by the end of the year.

But every suggestion/comment & concern is greatly appreciated :-) How are you at web-crawlers, VM? ;-)
Edit: if I have an efficient archiving web-crawler for the files/images themselves, I'd only need help getting the project info/comments... I think :-P

<...getting it>
               
               

               


                     Modifié par Rolo Kipp, 28 septembre 2012 - 04:38 .
                     
                  


            

Legacy_NWN_baba yaga

  • Hero Member
  • *****
  • Posts: 1944
  • Karma: +0/-0
The Vault Preservation Project
« Reply #5 on: September 28, 2012, 05:44:32 pm »


               I have a lot of hak content from the vault already that is relatively safe and soon I´ll buy another large HD for what i can start collecting more individual content (handpicked) from placeables, creatures, pc parts and tilesets. While collecting I can write a list with everything downloaded and so at least that stuff will be safe for the future or we sync up collecting things?

What i will avoid are collections paks and only gather material that once it is gone it is gone forever. The hard part will be about scripts and modules. Modules always use to much individual content which is almost impossible to gather and archive in a reasonable handy way manually.
               
               

               


                     Modifié par NWN_baba yaga, 28 septembre 2012 - 04:48 .
                     
                  


            

Legacy_NWN_baba yaga

  • Hero Member
  • *****
  • Posts: 1944
  • Karma: +0/-0
The Vault Preservation Project
« Reply #6 on: September 28, 2012, 05:50:52 pm »


               well i will do the creature part Rolo!
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
The Vault Preservation Project
« Reply #7 on: September 28, 2012, 05:54:24 pm »


               <encouraging...>

I will always encourage NwN fanatics to safeguard content individually, BY. Too many times I've found stuff that's vanished by asking "Anyone have...?".

I also have a lot of stuff (50 or 60gbs I think) archived. What I want to do is get it up in a backup site so others can access it/update it/add to it while the Vault is (temporarily, I do very sincerely hope!) in read-only mode. That means putting the projects together - info/files/screenies/comments).

So D/L as planned with my encouragement :-) But, if you have time after I get the site put together, you might also U/L your favorites to nwn-cc? All that stuff I have is bare files, no info/comments/screenies, etc. So I'll have to grab that stuff and U/L it myself. And, as I said, I'll be concentrating on the nwn1 haks.

<...certain types of greed>
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
The Vault Preservation Project
« Reply #8 on: September 28, 2012, 05:58:02 pm »


               <dancing...>

WooHOO! My *favorite* old witch! You rock!

<...with the old crone>
               
               

               
            

Legacy_NWN_baba yaga

  • Hero Member
  • *****
  • Posts: 1944
  • Karma: +0/-0
The Vault Preservation Project
« Reply #9 on: September 28, 2012, 06:15:48 pm »


               For the creatures i will also gather the original authors informations and 1 screenshot. Just tell us when your ready with the site so i can start uploading as well:)
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
The Vault Preservation Project
« Reply #10 on: September 28, 2012, 06:23:41 pm »


                <eyeballing...>

You can go register now (while I'm online and can approve you - all my old friends from here are *insta*-approved, I promise :-). The content type will be set up this weekend and I'll bump that here when it's live.

<...his calendar>
               
               

               
            

Legacy_nwnsmith

  • Full Member
  • ***
  • Posts: 125
  • Karma: +0/-0
The Vault Preservation Project
« Reply #11 on: September 28, 2012, 06:42:40 pm »


               silly question, but does noone have access to the http server?
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
The Vault Preservation Project
« Reply #12 on: September 28, 2012, 06:57:48 pm »


               <whips out his snitch-blade...>

Maximus does. I don't. According to the list of admins, we are it.
He *is* working on it, but he is paid full-time to work on IGN's other sites and he pretty much keeps NwVault up as a labor of love.

It is the thought of something happening to it, and the stagnation occurring right now that woke me up in a cold sweat this morning. I'm inherently lazy, but I love this game and this community. :-P

<...and points>
               
               

               
            

Legacy_NWN_baba yaga

  • Hero Member
  • *****
  • Posts: 1944
  • Karma: +0/-0
The Vault Preservation Project
« Reply #13 on: September 28, 2012, 07:03:08 pm »


               Registered! And well yeah, we better move now as to a date where it´s to late...
               
               

               
            

Legacy_Khuzadrepa

  • Sr. Member
  • ****
  • Posts: 347
  • Karma: +0/-0
The Vault Preservation Project
« Reply #14 on: September 28, 2012, 08:07:06 pm »


               I would like to help, Rolo. I'll get the 'Other' section, as there a lot of good tools in there.