Bioware forums are closing

[Baaleos]

               

The NWN 1 and 2 Forum scraping is now complete. (Took a few attempts)

Nwn 1 forum scrapes in at 400mb in size

Nwn 2 forum scrapes in at 282mb in size

 

I am not going to be hosting this for everyone to download, as I get charged for Download bandwidth.

So a million people downloading this from me will consume GB's of data transfer.

 

Can someone 'popular, well liked and trustworthy' contact me with a view to getting a download link off me?

               
               

               
            

[Legacy_werelynx]

               

Might I suggest uploading what you have as a project to new vault? From there any and everyone interested can download the raw data.

               
               

               
            

[Baaleos]

               

Suppose that could work - I will see if I can get that done shortly.

 

http://neverwinterva...her/save-forums

 

I've uploaded the 7zip compressed forum scrapes.

7Zip nicely compresses them both to around 17-25 mb.

               
               

               
            

[Legacy_Proleric]

               

I sent the link to OldTimeRadio for addition to the Omnibus when time permits.

               
               

               
            

[Baaleos]

               

Im going to do a bit of work to see if I can build an integration to transfer all these posts into an SMF forum,

Best case scenario - it gives us an interactive environment to continue conversations and threads.

Worst case scenario - it gives me a forum that looks popular with 100,000+ posts. Hehe

 

PS - I am running scraper against Dragon Age Inquisition Forums now.

Already spotted a niche part of those forums that broke the scraper - the tech support forum, did not go to a real forum, it goes to the support pages.

I have patches the tool to accommodate.

Also the Baldurs Gate forums behave differently too.

They don't do

 

Forum -> SubForum -> Post -> Topic

Its like the Baldurs Gate forums are the Sub Forums - so they have no main forum.

Which breaks the logic I built into the app.

 

I might be able to code it to test against the forums to determine which mode to run.

               
               

               
            

[Legacy_AndrueD]

               

It very gud to see sum1 doin so much work to keep our forum stuff going.  Old Bioware forum posts all got lost wen they shut that down cept few that ppl copy over.  Am thinking any newer NWN player will see how much NWN ppl care to have game go on so must be a very gud game for that to happen.

 

Tx very much to Baaleos and other folk to help do wut rest of us canna do. '>

               
               

               
            

[Legacy_henesua]

               

Someone should wget the whole thing when the site is in read only mode. This is the talk in IRC.

 

Its nice to have multiple versions of this floating around, but I think an archive dump using wget will end up being the canonical one. And I think that this is best to do when we can't add to the forums any more.

 

This will be simple HTML which is something that anyone can use rather than requiring a special DB or anything. What Baaleos is doing will likely have some advantages. I'm not dissing that. But I think just a bunch of files via wget will be accessible to far more people.

 

I'm not very technical either and so I understand what its like not to know what wget is. So here is a link about archiving a website using wget.

               
               

               
            

[Legacy_Snowdog65]

               

IMO there were two big blows to the community that we never really recovered from.

 

1: When Bioware killed the original Bioware NWN forums. Those were full of great content, and great people that never migrated.

2: When IGN killed their vault.

 

Maybe a lesson there about sticking with community sources, that won't get yanked as quickly.

               
               

               
            

[Legacy_Tchos]

               

It's a lesson about not keeping all your eggs in one basket.

               
               

               
            

[Legacy_Proleric]

               

IMO there were two big blows to the community that we never really recovered from.

 

1: When Bioware killed the original Bioware NWN forums. Those were full of great content, and great people that never migrated.

2: When IGN killed their vault.

 

Maybe a lesson there about sticking with community sources, that won't get yanked as quickly.

 

Quite agree about sticking with community sources.

 

Incidentally, did you know that most of the original Bioware forums are available in searchable form here?

 

Most of the IGN vault projects (minus comments) are available here.

 

These are community efforts to curate content that the commercial sites eventually gave up on.

               
               

               
            

[Legacy_niv]

               

Someone should wget the whole thing when the site is in read only mode. This is the talk in IRC.

 

That was me, and that's what I'll be doing once the site goes r/o. Nothing fancy, no reboot, just a static mirror just the way it looks when the time comes. It'll be hosted on nwvault.

               
               

               
            

[Baaleos]

               

It would be great if someone could develop a blockchain based website community.

One that is p2p processed and free from ownership, censorship and can never be taken down.

 

Not to be confused with TOR.

But more a system where the database for the forum posts and topics are distributed across multiple nodes - each node can be a person who downloads a program that then checks with peers for data etc

It then synchronizes the data, and the live website - could then query one or more peers for data that it determines a consensus on.

At the moment - bioware is the point of weakness here, because they just need to kill one server and maybe a db server, then we lose years of posts.

If those posts existed in the cloud, distributed across the whole community - it would be harder to kill.

               
               

               
            

[Legacy_Snowdog65]

               

 

Quite agree about sticking with community sources.

 

Incidentally, did you know that most of the original Bioware forums are available in searchable form here?

 

Most of the IGN vault projects (minus comments) are available here.

 

These are community efforts to curate content that the commercial sites eventually gave up on.

 

Yes I know an attempt was made to capture it. I have an account at the new vault.

 

But it isn't nearly the same, not nearly as navigable, and for the old forum capture is sterile, not live. Even if dead, a forum backup should be web accessible so it can be part of google search.

 

Those forums had many enthusiastic members, that didn't resurface when they went down.

               
               

               
            

[Legacy_Rogue1618]

               

It's a lesson about not keeping all your eggs in one basket.

 

You can keep them in one basket, just which basket is the issue.

Freedom Force transferred to a community run forum when the company shut down their site, been fine ever since with just one community run forum.

               
               

               
            

[Legacy_Tchos]

               

You can keep them in one basket, just which basket is the issue.

 

I know that's the prevailing sentiment here, as it was before, but I think it would be better to stop making the same mistake.