INFO: Draw calls not geometry as a bottleneck in NWN.

[Legacy_MerricksDad]

               

OTR:

 

I would be very cautious comparing how the toolset and the game actually work. They are certainly not the same, and I found that out the hard way while making my tree script. I certainly have no idea about the exact differences, so it may be just an issue of tight looping in one and of asynchronous processes in the other. What you have found so far though goes directly in hand with what I just ran into with the trees.

 

A placeable with 2000 separate meshes takes forever to draw in the toolset, and can even crash the explorer. However, for me, it runs just fine in game.

A placeable with 2 meshes, each with 1000 elements (which are just collections of face indices) has no issue at all in any one of the three programs.

 

It is definitely not about faces, it is about total object count and the number of draw calls as you said.

 

About emitters:

If you really want to try something and blow your draw count out of the water, make a model with an emitter that emits models with emitters. Don't loop it to itself of course, or it will instantly crash that stack, but make two separate models. I had tried this with a particle that dropped other particles in it's path. It was an utter failure. What I wanted was an ice comet that dropped sub-particles as it traveled, but I needed it to spawn particles at a different rate than the single particle math was allowing. On my machine, the timing between particle emission is set in intervals, not an actual fraction of a second based on the number of particles to emit per second. I suspect it was tied to the frame rate my video is allowed to run at. It certainly appeared different on my older desktop machine. Anyway, I was trying to force other math into the system and poke it a bit.

               
               

               
            

[Legacy_OldMansBeard]

               

OldMan'sBeard: If you can automate UV Mapping then it seems to me that combining meshes that use the same texture would be useful.

 

And what about automating texture atlases? Perhaps you could count textures which are smaller or equal to 512 x 512, and if you reach 4, these could be combined into 1 texture at 1024 x 1024, and each of the meshes that uses those textures could be remapped to the new texture.

The UV mapping wouldn't be a problem. Just multiply the U's and V's by 0.5 and add an offset of 0.5 if appropriate, depending on which quadrant of the new bitmap it's in. Software exists for stitching tga's together into bigger ones so that could probably be scripted too. There might even be something cunning inside the newer versions of 3dsMax that does it all for you (there's a lot of stuff in there that I've never got to grips with) or perhaps something could be done in Maxscript.

 

The question is: does it help? Does concatenating bitmaps into bigger ones decrease performance in the NWN engine, just because it's handling bigger bitmaps (albeit fewer of them)? Does it throw out mipmapping? (Suppose one mesh is further away from the camera than the other; the engine would want to mipmap it down but perhaps it can't if it's all the same bitmap?) I don't know the answer to this.

               
               

               
            

[Legacy_OldTimeRadio]

               

However, one thing we all need to remember is that NWN was designed over a decade ago. Rendering technology was not even close to what it is now and I can't believe that the more modern GPUs and CPUs we have today can't handle anything we have (and will have) thrown at them through the NWN engine. Time and again the engine has proven more robust than anyone can imagine and certainly has proven itself more than capable of meeting our needs.

 

Mostly agree except in the case of shadows, especially shadows on dynamic things.  For each triangle that's being shadowed, six more triangles are created and they dynamically interact with the rest of the world, including themselves.  Per light.  Most games do not use the true volumetric shadowing that NWN does.  NWN only gets away with it because of how low-poly the shadow volumes are.  Make a very high-poly sword, for instance, and shadow that high poly mesh and have enough of those running around (say, in a horde of goblins) and you will put more load on your GPU than (pick whatever the high-end game of the moment is).  No foolin'.  You can see all of this happening and look at the raw numbers by using gDEBugger and just ticking shadows on and off in the toolset.  Because NWN does such good shadowing, one can also read more about the volumetric shadowing in a number of scholarly articles written by game developers/computer scientists  which mention NWN by name and describe, in more detail, the effect.

               
               

               
            

[Legacy_Shadooow]

               

To be honest, I'm mainly trying to get the word out so that people (including me) might implement better practices going forward.  I haven't really spent much time thinking about how to retro-mod the existing content to be more efficient.  The only two instances are NPC optimization and VFX optimization.  Some of the Bioware effects create a mass of chunks and other things I suspect momentarily blow up the draw call count for the frames for which they exist.  Most of their effects are almost unnoticeable (given how briefly they exist) and at some point I might try to take an axe to them.

Oh yes this could be a way to fix the hellball crash!

               
               

               
            

[Legacy_OldTimeRadio]

               

Oh yes this could be a way to fix the hellball crash!

Might want to take a look at this, which purports to be an alternate/fixed Hellball effect.  I remember FunkySwerve mentioning something about someone from HG having fixed, in some capacity, the crashing issue with Hellball, searched and found the thing I link to in Werehound Silverfang's sig on HG.  Might want to follow up with Funky for more info.

               
               

               
            

[Legacy_OldMansBeard]

               

I'm almost tempted to install NWN and 3dsMax, just to explore this.

 

Well, almost. It's a bit scary.

 

We would need some test cases, a system of benchmarking, a team of people with various hardware to do the benchmarking, a discussion forum, somewhere to share files, a protocol for varying all the parameters that might be varied, someone to manage it all, ...

 

Don't look at me.

               
               

               
            

[Legacy_OldMansBeard]

               

What about an automated shadowbox creator?  Turn off shadows on rendered meshes.  Create duplicate, simplified, "fiercely optimised," non-rendered meshes that cast shadows, and, wherever possible, combine them with each other.  Would this help or hurt?  Would it even be possible?

 

I must say: when I started working on tilesets, I didn't see the point of shadowboxes at all, but I've since been won over to them.  As it stands, they're currently what I plan to do with my elven treetop city tileset.  The shadowboxes just don't need to be anything close to as complicated as the visible meshes, particularly since some visible meshes need to have extra faces solely in order to texture them in the way that you want, not because the geometry needs them per se.  This wouldn't be an issue for shadowboxes, which can be much simpler.

Este, there's a thinggy in 3dsMax for reducing polycount - I think it's called Pro Optimise or some such (not the standard Optimise modifier) - is that any use for making shadowboxes out of complex meshes?

               
               

               
            

[Legacy_Estelindis]

               

Este, there's a thinggy in 3dsMax for reducing polycount - I think it's called Pro Optimise or some such (not the standard Optimise modifier) - is that any use for making shadowboxes out of complex meshes?

 

If I recall correctly, both 3dsMax and GMax have optimise functions.  However, I probably never would have thought to use them for this purpose if you hadn't mentioned it.  Thanks!  '>

               
               

               
            

[Legacy_OldMansBeard]

               

I'm getting an idea for a test.

 

Build, say, a 12x12 area using a 3x2 tile group (one of the large ships, perhaps) repeated to fill the area completely.

Script a cutscene that moves the camera around the area.

Run the cutscene in game and record the frame rate (turn off vsync so it can go high as well as low).

Several people run that on their machines, so we get benchmarks for different hardwares.

 

Now, import the tiles in the group into Max and relink all the visible bits of mesh from all the tiles onto just one of the tiles, leaving the other tiles with just lights and walkmeshes. Export them all and run the main tile through CM3 to consolidate the meshes (Meshmerge-by-bitmap option turned on). Pop them into a hak and add it to the test module, so the original tiles are overwritten.

 

Run the benchmarks again. See if there is any difference.

 

Would that be a useful exercise ?

               
               

               
            

[Legacy_Valthrendir]

               

Este, there's a thinggy in 3dsMax for reducing polycount - I think it's called Pro Optimise or some such (not the standard Optimise modifier) - is that any use for making shadowboxes out of complex meshes?

 

The pro-optimize modifier does indeed reduce your polygon count. If I recall correctly it comes down to reducing your polygons/faces by a certain percentage. However, one downside is that most complex geometry loses its defined shape after a reduction around 75 - 80% (if you're working with lots of curved shapes, that is). If you're really picky in reducing polygon count, you might even try to transfer your model over to Blender and use the ''decimate'' modifier. Blender uses a different calculation method, thereby allowing your geometry to remain close to its initial, high-poly shape while still reducing your polygon count.

               
               

               
            

[Legacy__six]

               

How I would love to see source code for NWN.

 

Draw call being the bottleneck makes sense though. NWN's (presumably) using fixed pipeline stuff, so the only thing limiting how fast geometry can be processed, is how fast it can be sent off to the GPU. The physical distance it needs to travel across the hardware, and the number of times it has to travel there, is what ties down your framerate.

 

I find the draw order slightly curious insofar as it's sorting by texture, but not going so far as to actually combine the same-textured objects before drawing them. Which I'd love to try as a first optimisation step in a crazy alternate universe where we had the source code. Particularly for tiles, as that stuff is all static, and it'd be very easy to predict which tiles could be onscreen at any particular point.

 

I'm quite gratified to hear that less textures, more combined meshes is on the mark anyway, since I've kinda been assuming that's a good approach for a long time. And for reference, TNO is actually a really good tileset in that regard, compared to some of the standard ones. Waaaaaay too many textures though.

               
               

               
            

[Legacy__six]

               

I'd be interested to see how particle systems are constructed. I've found them notorious for slowing down the game, sometimes even apparently very simple effects.

               
               

               
            

[Legacy_MerricksDad]

               

I'd be interested to see how particle systems are constructed. I've found them notorious for slowing down the game, sometimes even apparently very simple effects.

 

I'd be more interested in creating a reuse algorithm inside the particle emitter system. One of the biggest issues I see with it, besides its create-when-needed methods, is that it doesn't clean up in a timely fashion. Still not as slow as firefox getting rid of page entities in memory after they are closed.

               
               

               
            

[Legacy_Rolo Kipp]

               

<still catching...>

 

I'm not quite ready to dive into this *very* tasty thread (busy with work :-/ ), but I'd like to mention a couple things to consider. Well, one thing to consider and one response to my name being dropped ;-)

 

@ MD: While I truly to want to develop procedural, chaotic world generation algos, I'm also very fond of *exceptions*. My original geometry engine proposal (to EA) in the 80's was a combination of DEM for world-mapping and 3D mesh exceptions with persistent deformable states. In regards to NwN, my plan (mentioned a few years ago) was to build areas procedurally *except* for special tilesets that would be constructed monolithically (like OTR's Megatron) and be both single purpose and highly optimized. (My descriptions before were considerably less specific and considerably more ignorant, but still on board with this idea.)

 

And that leads me to an item for consideration; Batching. That's what it's called in the jME engine, at any rate. This is done both at model creation and at runtime, sometimes even during the graphics update loop. Some of the most impressive terrain systems they are working on are actually calculating the geometry at runtime and batching mesh on the fly.

 

Joining Heightmaps thread

Marching cubes, triplanar mapping... thread and Awesome Video

(Edit: Had the wrong link on the Awesome video :-P Fixed now)

 

While the runtime option isn't available to us, an automated utility for examining an area, mod that could ultimately pull all the static mesh and textures (tiles *and * placeables) to produce optimized single use tilesets... that's intriguing :-) If we could still design and *build* with modular bits and pieces and then have a utility go through and optimize/compile as much as practical (starting only with tiles/groups/sets)... that could be amazing :-)

 

As for shadows in NwN... mostly I feel, like others, they just aren't worth the hassle in the current engine. In the *next* engine though... ;-)

 

<...the bug>

               
               

               
            

[Legacy_MerricksDad]

               

Are you in any of these channels where people are talking about the potential for a NWN 3? I hear rumors, but nothing more. I only have so many wishes left.