Any way to know what files NWN is loading, even from BIFs & HAKS?

[Legacy_OldTimeRadio]

               Here is my problem: I've got over ten different NWMain.exe files from various projects or things I've hex edited myself, about 35GB of content in my override directory and my whole installation is a mess.  My .HAK folder is also a nightmare and sometimes I have no real idea why I have a certain HAK in there.  It could have been a required HAK to login to a server once.  I have no idea.

What I'd like to do to solve this problem is to find some way to know all the files that NWMain.exe is loading and then use BifManager to make a set of BIFs with just the basic stripped down files that NWN needs.  Then I can basically "spawn" a new, small, copy of NWN and dedicate that folder just to one project.

And then have a bunch of little folders like that.  It would make my life easier for modding and I do not mind eating 50 (or whatever) megabytes for a "blank" NWN spawn each time.  It would be worth it, IMO, to not have so much clutter.  Then I can have elaborate setups where I have a custom NWMain.exe, custom BIFs, etc., each in their own folder as their own separate "project" to work on.

But I don't know how to get a list of every file NWN loads from inside a BIF to get to, say, the main startup screen.  Does anyone know how I can do that?

Thanks!
               
               

               
            

[Legacy_virusman]

               Stripping down base NWN files is not a good idea. I recommend using Link Shell Extension to create separate NWN folders with junctions/hardlinks/softlinks to shared data files/folders. That way, you won't waste space on copies of the same resources.
               
               

               

                     Modifié par virusman, 10 juillet 2012 - 08:34 .
                     
                  

            

[Legacy_Shadooow]

               NWN loads all bifs from data, then all erfs from texturepacks, everything from override, then module resources, then haks and lastly patch-haks

There is only one CC using bifs yet and that my community patch, this uses patch170.bif file.

There is about 300mb of unused content in NWN bif files, and if you are using my patch this is  raised by another 50mb at least. Now this content gets loaded by NWN and then overwrited, releasing this content would not only saved some HDD space but also increased loading speed. I was tkinking about this idea already but there isnt something like bif patcher to make it easy to install. Publishing whole modyfied bifs isnt really good idea. Also there are issues with languages, certain bif files are different across languages and thus there must be a special version for those languages otherwise it will crash the client (better idea is to unite files between all languages which is what Im planning to do for CP)
               
               

               
            

[Legacy_OldTimeRadio]

               @ Virusman - Thanks, I'll check that out.  From my notes, it looks like I tried doing something similar using fsutil.exe (probably for running something out of a small ramdrive but keeping the bulk of data on disk) and junction points, IIRC, but I ran into some limitation and couldn't pull it off.  This was four years ago, though, and it looks like Link Shell Extension does a lot more.  Do you happen to know if instancing an install of NWN like this, with Link Shell Extension, works?

Still, it's not my first choice of solutions.  That's really just based on my bad experience with fsutil and I will give LSE a try.  Do you happen to know off hand of a way if there is a way to see what files are being loaded from BIFs?  Something like FileMon (or equivalent) doesn't seem to cut it.  If something like that is really difficult, could I accomplish it with....IDA or something like that? 

@ShaDoOoW - Interesting.  BTW, do you think something like an instanced NWN could be used beyond testing, but for distributing content which might otherwise use a new BIF and the patch.ini in a player's main installation of the game?
               
               

               
            

[Legacy_virusman]

               

OldTimeRadio wrote...

@ Virusman - Thanks, I'll check that out.  From my notes, it looks like I tried doing something similar using fsutil.exe (probably for running something out of a small ramdrive but keeping the bulk of data on disk) and junction points, IIRC, but I ran into some limitation and couldn't pull it off.  This was four years ago, though, and it looks like Link Shell Extension does a lot more.  Do you happen to know if instancing an install of NWN like this, with Link Shell Extension, works?

Still, it's not my first choice of solutions.  That's really just based on my bad experience with fsutil and I will give LSE a try.  Do you happen to know off hand of a way if there is a way to see what files are being loaded from BIFs?  Something like FileMon (or equivalent) doesn't seem to cut it.  If something like that is really difficult, could I accomplish it with....IDA or something like that? 

Yes, I'm using LSE and it works easily and effective with NWN and any other software I need to copy or move to another HD. I'm using junctions in most cases.
I think I have a resource tracker for NWN in one of the experimental NWNCX builds, but I still doubt it'd be useful in this case. Removing stuff from bifs won't give you any significant performance gain, and you're much more likely to run into problems with missing resources. As for the HDD space, LSE seems to be a perfect solutuion to me.
               
               

               
            

[Legacy_Rolo Kipp]

               <trying to be...>

OTR, are you using this with the Multi-instance utility on the Vault? I've been meaning to explore both the hard links and the multi-instance thing for months. Hoping I can use it to experiment with running a seperate server and client on the same machine to set up a flexible MP/SP mod system... :-) Someday...

Thank you, Vm for reminding me, and for the link link =)
Anything to look out for when setting up the folders/links? Tricks for the shortcuts? Advice?

<...in two places at once>
               
               

               
            

[Legacy_OldTimeRadio]

               

virusman wrote...
Yes, I'm using LSE and it works easily and effective with NWN and any other software I need to copy or move to another HD. I'm using junctions in most cases.

Thank you for the confirmation.  I think this way I can get my regular modding mess covered and also be able to have a NWNX server setup, a NWNCX setup, and a NWNCX + NWNX setup with the bridge- if I'm understanding things correctly.  I have so much NWNX stuff I nolonger remember which folders and files were needed by each.

On a related topic (based on what ShaDoOoW said, or what I perceived he said), If I wanted to make an adventure which used NWNCX, maybe even as part of a server, what is the easiest way I could distribute that?  How could I make big changes to NWN and distribute something people could use...but without having an impact on their base install?  I mean, maybe they have NWNCX/NWNX (or whatever) installed as well- I don't want to step on their stuff with my stuff.  For instance, I have been growing to really think developing under NWNCX is not a bad way to go.  But I'd like to have some way to distribute the whole shebang, including the modified NWMain.exe and especially any plugins and stuff like that in a way that doesn't impact users.  Edit to be clear: I wouldn't be distributing serial numbers and without the player entering those any instance wouldn't work for multi or single player.  This would basically be the same level as the camera hacks starting around 2003 or whatever.

How do you think that could be done?  Would making a NSIS installer which snooped around, found the NWN installation, installed Link Shell Extender if it wasn't already installed, and then scripted a LSE instance of NWN (plus my modified files) be a way to get that job done?  Would there be an easier way?

I think I have a resource tracker for NWN in one of the experimental NWNCX builds, but I still doubt it'd be useful in this case. Removing stuff from bifs won't give you any significant performance gain, and you're much more likely to run into problems with missing resources. As for the HDD space, LSE seems to be a perfect solutuion to me.

You're probably right, so I'll take that route first.  I still like the idea of one day experimenting with a stripped installation like that- maybe the resource tracker functionality will be in a future build I can play with.

@Rolo - You know, I took a look at that for what I wanted to do but I wound up dismissing it for what I wanted for some reason or another- and maybe without merit.  Now that I'm thinking about it, I'm wonderinf if this concept has some kind of applications outside of modding but distribution of modules...modules which heavily rely on third-party systems like NWNCX to pull off some new functionality.
               
               

               

                     Modifié par OldTimeRadio, 10 juillet 2012 - 06:06 .
                     
                  

            

[Legacy_Rolo Kipp]

               <climbing out of...>

OldTimeRadio wrote...
@Rolo - ...Now that I'm thinking about it, I'm wonderinf if this concept has some kind of applications outside of modding but distribution of modules...modules which heavily rely on third-party systems like NWNCX to pull off some new functionality.

We're sniffing the same dinner, OTR ;-)
Now we (community) just have to cook up some stone soup ;-)

NwN is a very selfish/self-centered little engine. It needs to be rather pampered. So far that burden has been carried by every category of user, from the builder to the player; D/L this and that hak, be sure versions match. Put this in your override. This in your tlk. Be sure to apply this patch to a vanilla version of the exe.  If we can standardize, modularize, or even solarize distribution/use methods for the system, then we can start doing the same, with optimization thrown in, for content.

Again, we need to divorce content from systems and make installation/enhancement/modification just as easy as we can for the "consumer". 

All of this is possible. Just may not be possible for *us* :-P But I'm not giving up anytime soon.

<...an old, deep rut>
               
               

               
            

[Legacy_virusman]

               As an alternative to LSE, you can just copy nwmain.exe, binkw32.dll, mss32.dll, nwn.ini, nwncdkey.ini, nwnplayer.ini to a new folder and change (some or all) aliases in [Alias] section in nwn.ini to point to the original NWN folder. It works, I've tried.
This is probably the better way for distribution as it doesn't rely on 3rd party tools and FS modification.
You'll be able to modify the binaries in any way without touching the original game files; you'll be able to customize some resources (modules, haks, override), while using some of the original files from the NWN folder(data, texturepacks, etc.)
               
               

               

                     Modifié par virusman, 11 juillet 2012 - 02:40 .
                     
                  

            

[Legacy_OldTimeRadio]

               Weird!  That's one of the first things I ever tried and I could have sworn it didn't work.  Hrm...What the heck?  It could have been before 1.69 came out, it was so long ago.  Nomatter, I'll try that too.  Thanks for the wisdom, VM!
               
               

               
            

[Legacy_OldTimeRadio]

               

Rolo Kipp wrote...
We're sniffing the same dinner, OTR ;-)
Now we (community) just have to cook up some stone soup ;-)

I think we are, anyway- heh heh!  After stewing on this for a couple of days what I want my magic wand to do is make it so:
* Custom content creators can start chopping up NWN a bit more agressively than they have in the past with a way that would normally affect user installs adversely
* PW/Single player module creators could distribute modules or setups which would normally affect user installs adversely
...without affecting those user installs so adversely.

Maybe some other stuff, too.  VM might've given the solution or the outline of a solution that would work.

  My example, kind of thinking of this off the top of my head, is...something like...let's say Vuldrick when he was making his popular universal base module, leaned heavily into NWNX/NWNCX.  Then, flat out lost his mind and started making changes to the BIFs, replacing all sorts of stuff because he wanted to so he could...override the default campaigns?  Universally change scripts?  Whatever, I dunno.  Even hacked the NWN updater so it would snarf down updates specifically for that project.  Vuldrick, if you're out there I apologize for using you as an example, BTW!  This is purely hypothetical.

So that's like all kinds of kooky "do NOT touch" stuff thats not only been touched, it's been molested, folded, spindled and mutilated into something...completely new and dreadfully, unquestioningly, sexy.  Sexxxy.  Maybe he's so talented he changes every bit of OC content to have a wild west, gunslinger theme.  Or star pirate erotica.  "Total Conversion" kind of stuff.

How do you distribute something like that in a way that doesn't shoehorn every user into those changes?  How do 5 different people/groups/persistent worlds/whatever distribute such wildly disparate stuff in a way a player can enjoy them all...on the same machine at the same time with the absolute least possible hassle?  Again, it seems like the two approaches VM gave are probably either good starts or even maybe full solutions that could be automated in some way.  Possibly by as little as a batch script, I dunno.

BTW, I should ask, are we smelling the same dinner on this?  If not, what is the dinner you're smelling?
               
               

               
            

[Legacy_henesua]

               Whatever dinner you guys are sniffing, I like it.

Unfortunately I'm still eating Top Ramen.
               
               

               
            

[Legacy_virusman]

               It's possible to cook up a NSIS installer script that'd get the NWN directory from the registry, copy some files to its own folder, edit nwn.ini, patch the binaries with VPatch, copy content files, etc.
               
               

               

                     Modifié par virusman, 13 juillet 2012 - 05:26 .