NWN Gamespy Server Replacement Project

[Legacy_Zarathustra217]

               Ah, yeah, md5 as optional is surely the way to go.
               
               

               
            

[Legacy_Skipper24]

               So gamespy pulling the plug and the gamespy chat/server lists for NWN 2, going down has also effected Nwn1 aswell?

This is terrible.

I dunno anything as far as putting something together to fix the problem/replace it but I know this much, people still buy Nwn1 and 2. What ever solution you people, you the ones who will hopefully take up this challange, with the knowledge skill and expertise to do so, will take into account, a new player will  click multiplayer, go through the login thing and then realise there is no connection too the server listings.

IF you want both games to continue to survive you need something to come up so these new people know, were to go to find servers or a specific server and so forth. Otherwiset they'll turn away from the game and that doesn't help.

What ever the solution is.. the dev's of either game should hopefully step up and do something so then the player community can take over.

It sucks but this is exactly what happend to one of the best space sims, Freelancer. Microsoft hosted the global server listings when that went down.. new players didn't know what to do.
               
               

               

                     Modifié par Skipper24, 08 décembre 2012 - 12:30 .
                     
                  

            

[Legacy_Zarathustra217]

               Yeah, it would be preferred if we could make something basic relatively fast so the current players don't abandon the game.

As for the actual coding, any proposals for a development framework? Have anyone had any experience developing with Mono? http://mono-project.com/
               
               

               
            

[Legacy_Melkior_King]

               The simplest solution (from the player/server point of view) is for someone to make a server which does the same thing as GameSpy (except only for nwn) and give us the IP address.  We can then enter that IP address along with the name of the GameSpy nwn server into our local "hosts" file to redirect the nwn GameSpy calls to the new site.

The adding of the IP address and server name to the local "hosts" file could easily be automated.  Distributors such as gog.com could add the code to the installer for nwn Diamond Edition.

I expect it would take me only an hour or two to cobble together a "hosts" file updater to do the same job and to make it foolproof (so the same entry doesn't get added twice or more).

It might take me a day or two to put together a version which would download the information from a known location and update it automatically, if that's an option.
               
               

               
            

[Legacy_Visavant]

               

Melkior_King wrote...

The simplest solution (from the player/server point of view) is for someone to make a server which does the same thing as GameSpy (except only for nwn) and give us the IP address.  We can then enter that IP address along with the name of the GameSpy nwn server into our local "hosts" file to redirect the nwn GameSpy calls to the new site.

The adding of the IP address and server name to the local "hosts" file could easily be automated.  Distributors such as gog.com could add the code to the installer for nwn Diamond Edition.

I expect it would take me only an hour or two to cobble together a "hosts" file updater to do the same job and to make it foolproof (so the same entry doesn't get added twice or more).

It might take me a day or two to put together a version which would download the information from a known location and update it automatically, if that's an option.

This is a working server that is receiving gamespy updates from one of the NWN1 servers talking to it, the only edit was the hosts file change. The output you see on screen is to put the data also into an XML format to output to a webpage so that we have listings immediately seen on a website while I work on the game client's internal listing.

I've written an application that patches nwmain and nwserver for both NWN1 and NWN2, but also gives people the option to simply click a button and it modifies only the hosts file.
               
               

               

                     Modifié par Visavant, 08 décembre 2012 - 03:14 .
                     
                  

            

[Baaleos]

               I was bored, so came up with a C# App that uses the cached/legacy info from the master server.
Using this app, players will have visible access to about 200-300 servers.

The info coming from here.
https://dl.dropbox.c...mespy_cache.htm

If we had a community website to point it at - we could have more updated info.


http://nwvault.ign.c....Detail&id=1540

The application and Source has been uploaded to the vault - awaiting approval.
Note - the app was coded very fast, so its bound to have bugs.
But the general usage is

Turn it on, wait a few seconds, and it should populate with server rows etc

Red Rows = Offline Servers, or Servers that just couldnt be communicated with.
Green = Online Servers

The Button on the right will perform a few actions.

1. It will look for the nwn install directory in the windows registry - some windows permissions may affect this - run as admin if u get errors.
2. It will start NWN up and give the server address to the application.
3. It will use a Memory class I wrote for my RhunDownloader application, this class modifies the nwmain.exe in memory, to disable the MasterServer authentication - so you should be able to jump right into the game with no wait time.


Some of those functions require elevated privilages - works without issue for me.



Note - Im not sure why, but some servers were not responding to the UDP query at all.
Which is odd - cause other servers would return a message to say offline or some sort of exception.
In the case of the weird ones - they just froze the udpClient receive method.
As result - I had to implement a 10 second max timeout for each server query.

If you see the server updating seemingly freeze, its just cause I have the max threadcount at 5, once the 10 seconds per thread timesout, it should free up threads and continue updating the table info.


Until the vault entry is approved u can get the binary from here.
http://31.132.2.110/...erverJoiner.exe

Note - I cannot promise to be making frequent updates to this, this is just meant as a proof of concept really.
Plus I was bored - so I made it to occupy myself.
               
               

               

                     Modifié par Baaleos, 08 décembre 2012 - 05:11 .
                     
                  

            

[Baaleos]

               I just read the previous posts about a community updater/loader application?
Sounds interesting.


My own 2 cents on the subject (since its something I've been using for 2 years or so)

I upload my files to a webserver, (a folder on the webserver)
I call a URL when finished - with arguments pointint to that folder
It processes all the files in the folder.

It takes info such as

FileName,
Extension,
Size in bytes,
md5

It compares the md5 against any already existing entries in the database.
Updates as necessary.

Then, once per hour or when triggered manually,
another php file turns the mysql data into an xml

<?xml version="1.0" ?>
- <Files>
- <File>
 <FileName>219.bmu</FileName>
 <Size>2561160</Size>
 <Date>1277584559</Date>
 <MD5>ee61da9193b63333ca03df4ac6259bf3</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>ffxiii_dust2d.bmu</FileName>
 <Size>3662064</Size>
 <Date>1277089435</Date>
 <MD5>0d1c1af6665dce0d41e2f2e4f01fa30e</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>229.bmu</FileName>
 <Size>3305545</Size>
 <Date>1277584693</Date>
 <MD5>7003eda8ea94bf0af41a501e2bb8b85c</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>202.bmu</FileName>
 <Size>2720820</Size>
 <Date>1277584375</Date>
 <MD5>1ae1cd90576c39d132a4a62daa842c96</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>mus_def_sluagh.bmu</FileName>
 <Size>3831671</Size>
 <Date>1277089722</Date>
 <MD5>afa680c09d688f1ecee261efe074cc09</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>212.bmu</FileName>
 <Size>5256997</Size>
 <Date>1277584523</Date>
 <MD5>03cc78ce65a79a9d134ca09077fd5d56</MD5>
 <DELETE>FALSE</DELETE>
 </File>

This XML is downloaded onto players machines, then parsed by my updater application.
Files that are marked as Delete = true  
are ones that dont get downloaded, and get removed if they exist.

MD5s that dont match, are redownloaded fresh
And the file extension determines the location towhich they are transferred.

Although, in my case, I have some strong customizations.

eg - Instead of downloading wav files, I download mp3 files, then use ffmpeg - bundled in the executatable, to convert to wav.
Result is that it shortens download times - a wav file would be abour 3-4 mb for 30 seconds of audio or less,  but ends up being 300-500kb as mp3.

I also recently introduced rar capabilities - that allows me to bundle override content, into a rar file - it gets downloaded, unrarred, and content transferred to override.

Also same for tlk files -  I made a new extension called
rtlk  (rarred tlk files)

The Cep2.3 tlk file being 7mb or so - ends up being about 400-500kb when rarred.


Now - in my case- I did alot of funny steps there, such as using the override directory etc alot, because my application is aimed towards allowing people to enter without needing the haks.
Hak content = compulsory
Override = Voluntary -(they might not see or hear the content, but they can still enter the server)


Hope this gives you guys some ideas?
               
               

               
            

[Legacy_Zarathustra217]

               The issue with using just regular C# is that it won't immediately support other platforms. NWN is one of the few major games that'll both run on Mac and linux and we shouldn't leave them out.

And yeah, the utility should definitely have at least zip capability (though rar is of course preferred) because most NWN content compile incredibly well.

I fundamentally believe though that our highest priority should be to make it as simple as possible for the end users (both players and content providers). NWN is an aging game and if there's too much hassle, it'll easily scare off people. If we can avoid manually editing the host file, I feel we should aspire to. By same reasoning, making file size, date and md5 optional makes it simpler.

In terms of how to manage content, wouldn't it be far the simplest solution if any package should have the same folder structure as NWN, so it simply extract to the NWN directory? (e.g. so that hakpaks are in the hak subfolder, talk tables in the tlk, override files in override etc.)

Also, I think it might be a good idea to allow for specifying some content as optional - so a server may have an optional override, music or portrait pack.
               
               

               
            

[Legacy_leo_x]

               Sounds kind of like you rolled your own rsync?  I wonder ...if you could use nwnpatch.ini to load a hak client side while having the stuff in the override on the server or if client would ignore them if they weren't in the servers hak list.  Not sure if that would even make anything more manageable tho.

Don't know if I will bother working on anything since it looks like it will be covered.  I just wanted to be apart of the voices that say this is opportunity to make something better, something that makes every server and all the great custom content out there one-button accessible to players.  My old nwnx_haks plugin was on kind of the same lines.
               
               

               
            

[Legacy_Zarathustra217]

               Oh, and a strong favouring point of not simply modifying the binary or using a custom host file is that a community made launcher would allow for auto-downloading and general content distribution.
               
               

               
            

[Legacy_Rolo Kipp]

               <*snoring softly*...>

Baaleos wrote...
...
If we had a community website to point it at - we could have more updated info.

http://nwvault.ign.c....Detail&id=1540

The application and Source has been uploaded to the vault - awaiting approval.
...

<boss! boss! wake up!>
*snort*...*grumble* <get *up* old man!>
*mumble* *grumble* mummy... <sheesh... *wing-slap*>
What! Where! I didn't do it! <that's the point *raven scowl*>
What are you blathering about now, bird. Where's my slippers? <i'm a raven, not a puppy. get up>
Why? Where's the fire? <not under you, evidently. big b has a project to approve on the vault>
Well, why didn't you *say* so?! You *mumble* that's what the problem is <*beak gapes in astonishment*>
What? <nothing, wizard>

<...and dreaming of good things to come>
               
               

               

                     Modifié par Rolo Kipp, 08 décembre 2012 - 06:04 .
                     
                  

            

[Legacy_Visavant]

               Doing something similar for another community as well, though I found that md5 checksumming on client machines with multiple files exceeding 200 MB to be problematic with older hardware. Serverside, I would still calculate the md5sum, though I entered in the file size and last modified entires which the auto downloader would launch on, only using MD5 when there was known corruption in a file.

Baaleos wrote...

I just read the previous posts about a community updater/loader application?
Sounds interesting.


My own 2 cents on the subject (since its something I've been using for 2 years or so)

I upload my files to a webserver, (a folder on the webserver)
I call a URL when finished - with arguments pointint to that folder
It processes all the files in the folder.

It takes info such as

FileName,
Extension,
Size in bytes,
md5

It compares the md5 against any already existing entries in the database.
Updates as necessary.

Then, once per hour or when triggered manually,
another php file turns the mysql data into an xml

<?xml version="1.0" ?>
- <Files>
- <File>
 <FileName>219.bmu</FileName>
 <Size>2561160</Size>
 <Date>1277584559</Date>
 <MD5>ee61da9193b63333ca03df4ac6259bf3</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>ffxiii_dust2d.bmu</FileName>
 <Size>3662064</Size>
 <Date>1277089435</Date>
 <MD5>0d1c1af6665dce0d41e2f2e4f01fa30e</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>229.bmu</FileName>
 <Size>3305545</Size>
 <Date>1277584693</Date>
 <MD5>7003eda8ea94bf0af41a501e2bb8b85c</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>202.bmu</FileName>
 <Size>2720820</Size>
 <Date>1277584375</Date>
 <MD5>1ae1cd90576c39d132a4a62daa842c96</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>mus_def_sluagh.bmu</FileName>
 <Size>3831671</Size>
 <Date>1277089722</Date>
 <MD5>afa680c09d688f1ecee261efe074cc09</MD5>
 <DELETE>FALSE</DELETE>
 </File>
- <File>
 <FileName>212.bmu</FileName>
 <Size>5256997</Size>
 <Date>1277584523</Date>
 <MD5>03cc78ce65a79a9d134ca09077fd5d56</MD5>
 <DELETE>FALSE</DELETE>
 </File>

This XML is downloaded onto players machines, then parsed by my updater application.
Files that are marked as Delete = true  
are ones that dont get downloaded, and get removed if they exist.

MD5s that dont match, are redownloaded fresh
And the file extension determines the location towhich they are transferred.

Although, in my case, I have some strong customizations.

eg - Instead of downloading wav files, I download mp3 files, then use ffmpeg - bundled in the executatable, to convert to wav.
Result is that it shortens download times - a wav file would be abour 3-4 mb for 30 seconds of audio or less,  but ends up being 300-500kb as mp3.

I also recently introduced rar capabilities - that allows me to bundle override content, into a rar file - it gets downloaded, unrarred, and content transferred to override.

Also same for tlk files -  I made a new extension called
rtlk  (rarred tlk files)

The Cep2.3 tlk file being 7mb or so - ends up being about 400-500kb when rarred.


Now - in my case- I did alot of funny steps there, such as using the override directory etc alot, because my application is aimed towards allowing people to enter without needing the haks.
Hak content = compulsory
Override = Voluntary -(they might not see or hear the content, but they can still enter the server)


Hope this gives you guys some ideas?

               
               

               
            

[Baaleos]

               It is true - calculating md5's can be tricky - in order to calculate it, you need to read the entire files contents.
Reading 200mb +  files x (amount of files)
could cause problems.

Unfortunately, I cant think of any other way of detecting changes in the files besides the md5, or some sort of relational version tracking similar to how svn or cvs works.

Keeping track of file size 'might' be sufficient - if we are talking about bytes, then chances of us having 2 versions of the file with exactly same file size are rare.
               
               

               
            

[Legacy_Visavant]

               We compared on the combination of filesize and last modified before resorting to md5.
               
               

               
            

[Legacy_tzaeru]

               Oh hai, seems I'm promptly killing my 8 year lurking spree, but here goes:

Melkior_King wrote...

The simplest solution (from the player/server point of view) is for someone to make a server which does the same thing as GameSpy (except only for nwn) and give us the IP address.  We can then enter that IP address along with the name of the GameSpy nwn server into our local "hosts" file to redirect the nwn GameSpy calls to the new site.

The adding of the IP address and server name to the local "hosts" file could easily be automated.  Distributors such as gog.com could add the code to the installer for nwn Diamond Edition.

Is the GameSpy IP in a data file? If it's directly in the executable, then the closest to a legal way to achieve that is to intercept the connection attempt to GameSpy's server and modify the IP on the fly.. But even that might fall under reverse engineering and be promptly illict, except if you happen to live in a country where reverse engineering for the purpose of maintaining compatibility and functionality is legal.

Baaleos wrote...

It is true - calculating md5's can be tricky - in order to calculate it, you need to read the entire files contents.
Reading 200mb +  files x (amount of files)
could cause problems.

Unfortunately,
I cant think of any other way of detecting changes in the files besides
the md5, or some sort of relational version tracking similar to how svn
or cvs works.

I think the best way is to maintain version history in a data file and just do the baseline installation from scratch when the launcher/updater software is used for the first time.

So, for example, a simple resource for the files could look like this:

-1.1:
hak/some_server.hak
-1.0:
hak/some_server.hak
hak/cep_stuff.hak
tlk/some_server.tlk

Once the baseline installation (version 1.0) is done, the launcher/updater just marks down the installed version and when a new version is in, installs the files from that.

For convenience, there can be a simple options checkbox for telling the launcher/updater to assume that base installation is currently in so old players don't need to redownload everything.
               
               

               

                     Modifié par tzaeru, 08 décembre 2012 - 11:04 .