Performance: ASCII Vs Binary.

[Legacy_Jedijax]

               After all the trouble I've been through (See Creatures Freeze-framing, and Return of the Freezing Creatures), I decided to REPEAT every trouble-shooting test kindly suggested by community members, since I changed my ATI 3870 for a significantly better Nvidia GTS 250, and the results were bound to be different. I was right.
The main difference is whenever the override folder was empty and there were no haks involved, the freezing problem caused by creatures spawning/coming into view was clearly diminished; oh, it is still present, but with a radically shorter time-span.
So, I tested and tested and tested, and found out the high-res textures, modified WOK, 2da, NCS, ITP, UTP, PLT, etc. files cause no increase in the freeze-framing issue. This leads to a simple conclusion: The lag increases when high-polys come into play.
Now, this may seem obvious to most, but it wasn't really that clear with my older card, which showed freeze-frames of absurd duration even on clean installs of the game.
Now, I usually have about 200 megs of high-polygon models in the override folder alone, and who-knows-how-many more in personal haks, aside of CEP and the like. My NWN looks gorgeous, up to challenge any modern game. The whole point of this customization is to make the game look great, since, let's be honest, the original models were already quite horrible back in 2001, when it came out. What's the point of being able to customize the thing to suit higher standards if it becomes unplayable? I'd rather uninstall it if I had to play it "raw", without model overrides.
Anyway, some have suggested turning models into binaries actually may increase the performance of the game, since, supposedly, they are easier to read. The question is, does anyone have any real experience in this regard? If so, could you explain thoroughly what the difference was in your case? I mean, changing tons of models from ASCII to Binary is no easy and quick feat, and I don't want to do it if there's no real upgrade in perfomrance... Thanks in advance!
               
               

               
            

[Legacy_Jez_fr]

               Well I can't explain exactly the technical reasons, though I guess them, but yes binaries are quicker to read by the program. Thats a fact I've seen by converting "high" poly model from recent games.  to NWN. The higher the poly number, the higher performance gain compiled model provides.

I suppose the CEP guys saw the same thing, and why then went through the trouble of compiling everything again.

               
               

               
            

[Legacy_Jedijax]

               OOOOhh, so you would recommend the change, Jez? If so, what software would you use? I used to be fond of Carottes for it's bulk-conversion options, but it doesn't seem to be working anymore under windows 7...
               
               

               
            

[Legacy_Lord Sullivan]

               The technical difference is quite simple really, when you use ASCII models the game as to convert them
to binary when it loads the models and when the models are already binary there is no conversion
occurring.

Performance could be affected on module/areas where allot of ASCII models are used, but not to
confuse with having allot of ASCII models available in your "\\\\override" folder or a hakpack. If there are allot of ASCII models in the "\\\\override" that well... "Override" commonly used game resources it would in fact stress the system a bit.
               
               

               

                     Modifié par Lord Sullivan, 20 novembre 2010 - 06:07 .
                     
                  

            

[Legacy_Jez_fr]

               Well I never did comparison between compilers so can't tell if one is better than another. I just use old NWNmax compiler (I think its Torlack's one) if 1 by 1. For batches/bulk I only use this one http://nwvault.ign.c....Detail&id=343.

Maybe ask over at Custom Content forum instead?

               
               

               
            

[Legacy_Jedijax]

               Lord Sullivan: Well, what I meant is if the creatures appearing are ALL override models with high polygon counts, wouldn't there be a performance increase? You see, in my recent tests, overriding creatures increased the freeze-framing issue. So, you wouldn't recommend a complete "binarization" of my models?

Jez_fr: Thanks for the link, Jez, I'll check it right away!... You know, sometimes I spend so much time troubleshooting NWN and visiting forum pages related to it, that I wonder how my life came to this... It's really sad...
               
               

               

                     Modifié par Jedijax, 20 novembre 2010 - 07:27 .
                     
                  

            

[Legacy_Lord Sullivan]

               I'm not "NOT" recommending a binary conversion, technically it should help that is for sure.



As for high poly models, that depends of how much higher we're talking about. If we're talking

from 3000ish polys to 4000-5000ish... it's not that much higher and we're only talking a bit higher than the standard, but if where talking more then double the standard model, then we're starting to talk higher poly models.

               
               

               
            

[Legacy_Jez_fr]

               rethinking,  Jedijax, I bet that @200 meg your override is more full of TGA and PLTs than models (they are rather lightweight). Why don't you put all that in a hak, with TGA converted to DDS, they are faster (well usually when I say that, there's a know-better showing up and saying the opposite XD ), and I bet you have a lot in your override?

Even if you play a module with its own haks, you can add your "override" hak on top of the module's hak list, cancel the build, save, and you have you precious override as a hak instead.

Man, you shouldn't have anything in your override. I know its common place, but override should be used only for testing nothing else. Thats a Bioware's rule since day one. Why don't people listen eh?
               
               

               

                     Modifié par Jez_fr, 20 novembre 2010 - 08:38 .
                     
                  

            

[Legacy_Jedijax]

               Gotcha Lord Sullivan! I just used Carottes to convert all my override models into binaries, and about 30 from the 4000+ files couldn't be turned by the compiler (actually it is merely a front-end, a GUI for Torlack's thingie). I did notice the freeze was shorter, but since most of my creatures are in haks, I'm thinking it will take way longer to turn everything into binaries, and test thoroughly to gain any significant data!

On the other hand, Jez, you're right, the 200 megs were merely model files, the other 900 are a mix of dds, tga, 2da, ptl, etc files. However, the Hak Vs Override issue is something I've discussed with lot's of community members; opinions are quite divided on what is best for performance. Back when I had about 800 megs in override, I was concerned about such, but Cervantes told me he had well over 1 gig, and the game ran silky smooth. So, seeing as most custom modules use haks, I decided to divide my resources; I have 1.18 gigs in override and 852 megs in haks. This haks are at the top of ALL my modules; I go as far as to change about 50% of a module in order to work with my haks.
Also, I had heard having too many haks on a module was counterproductive; perhaps you could clarify this for me?
The DDS matter interests me. I use tga's mostly because I can see them as thumbnails easily in the explorer, without opening an image editor or viewer (yes, I do A LOT of texture editing too... I have no life...), and recently, I get full-size previews of tga's merely by placing the mouse pointer over the file. I am willing to turn them into dds, but there are two issues: First, I don't know what software may do that, and second, Would it really make a difference performance-wise?

Last, but not least, in addition to the thread's main topic about ASCII and Binaries, maybe we could resolve, once and for all, which is best: Hak Vs. Override. I've gone for the "50/50" approach, but what do you guys think?
               
               

               

                     Modifié par Jedijax, 20 novembre 2010 - 09:46 .
                     
                  

            

[Legacy_Jez_fr]

               wow 1.18 gb in override Oo nice =)
It's amazing how differently one can use NWN.
About the number of hak, sorry, I have no hard evidence one way or another, but as far as performance goes, I think its really secondary compared to actual impact TGA/DDS have (or big mdls). I never bothered with that.
On the other hand, some popular tilesets have very big TGA (like 1024x1024), if you convert those to .dds, you'll feel a difference, no doubt (depend on the area and number of textures loaded in memory of course). I convert systematically all textures to dds, whoever did the original pack (well I resize some of them even...), some authors aren't reasonnable ^^
But you have a point, about editing. Converting to dds, and then to back to TGA etc... is a pain, and you lose a little on quality every time you do it.
My best advice would be, as soon as you're finished editing something, convert it to dds, then move it to a hak. and keep in the override only those you'll edit.

About the Hak vs Override debate, I'm just trusting Bioware original team, in all cases (even if some of them did some stupid things, that happens in all game dev team, the fact remains NWN is a gem and THEY knew what they were doing. It'd need actual performance numbers and stats to convince me otherwise, not speculations). Just my personal preference/experience.

(Just to be a pest, this topic is hardly Tech support relevant , its custom content really ^^)
               
               

               

                     Modifié par Jez_fr, 20 novembre 2010 - 11:28 .
                     
                  

            

[Legacy_Jedijax]

               Well, I thought the same thing back when I started it, but since it is meant to discuss the impact of conversion of models (and now even textures) on game performance, I decided it is sufficiently relevant from a technical point of view. I mean, the objective of this topic is to solve a very specific, technical issue, one directly related to game performance, and as you know, we've covered a VERY wide spectrum of possibilities trying to solve it.



By the way, what tool/app do you use to convert TGA's into DDS? You've convinced me to do those too, and since I'm systematically sweeping through all of my added resources... Can it be done "in bulk", or do I have to do it file by file?
               
               

               
            

[Legacy_Jedijax]

               AAAAAAAAAAAAAAAAAAAAHHH!! I converted my TGA's into DDS and it messed up LOTS of tilesets and placeables, so it's a no-go. Besides, I didn't notice any change in the freeze-frame issue. I think I'll refrain from such conversion!



Now, I made extensive tests in a module with no haks, and the freezing issue persists whenever a creature spawns or comes into view. Although the binary conversion diminished the freezing very slightly, the problems is still there, so it seems it cannot be solved by converting models... sigh... guess this particular venue is going nowhere near to solving my dilemma.
               
               

               
            

[Legacy_Jez_fr]

               The TGA to DDS I use should be this one.

Be aware that TGA must not be RLE compressed for this to work properly (and Im not sure NWN likes that either), otherwise the dds come up fooked up (sort of greenish grid. You can check your dds with NWNExplorer).

What happened that "broke" your tilesets?
               
               

               

                     Modifié par Jez_fr, 20 novembre 2010 - 08:47 .
                     
                  

            

[Legacy_Calvinthesneak]

               Ok first off, have a look at this thread, I had to learn about binary vs ascii once upon a time
Binary model Introduction


Then read this one ASCII vs Binary comparisson
               
               

               

                     Modifié par Calvinthesneak, 20 novembre 2010 - 09:07 .
                     
                  

            

[Legacy_Jedijax]

               Fooked? That sounds funny... Sorry, I'm mexican, so maybe the word is lost on me, or I'm just too dumb to get its meaning in context? Hoho! Yes Jez (See? I can rhyme!), many of the textures simply stopped showing, as if the models didn't recognize them; on the other hand, there are many, MANY tga's used for GUI elements and inventory icons, which didn't seem to agree with DDS.



Hey there Calvin! Thanks for the links, I'll take a look right away! By the way, I hope you could tell me a bit about your game experience under the GTX 295, since there may be a remote but possible chance of me getting one!