Of the following, which do you think are going to have the most FPS impact on a tileset?

[Legacy_OldTimeRadio]

               

Every once in a while, I go walking around a random empty tileset with FRAPS or the "trace fps" console command on and try to figure out why certain areas or vantage points lag more than others.  Then I maybe compare their performance to another area in another tileset.  Since it's rarely an apples-to-apples comparison, it's not so easy to come to clear conclusions.

 

Of the following list of culprits, do you have any thoughts which of these have a higher (or lower) impact on FPS than others?

* Improper visibility nodes for a given tile in an exterior tileset.  (i.e. something other than "A" should be used, but isn't)

* Meshes using the same texture in a tile are not combined.

* Number of tile lights in a given area, even when inactive.  (i.e. many tiles' lights compared to one giant tile group (of the same size) which only get's one tile's worth of lights/animations)

* Tiles whose nodes are not ordered in any way.  (i.e. AABB, geom, anim node and lights not intentionally ordered in some way in the model's hierarchy)

 

That last one is sort of interesting in its own right.  Looking at tiles and tile groups getting drawn in an OpenGL debugger, sometimes it looks like the game has a really good grasp of what order things should be drawn in and other times it appears to be "randomly" throwing items up on the screen.  Still not quite sure why some tiles tip off the engine better than others, but one possibility is node ordering.  From a few random tests (they take quite a bit of time to run), tile groups have been curiously solid as far as getting things drawn in the right "order".  I'm not sure how much of an impact that would have but if it does I imagine it would have the most impact when lights or alpha meshes are drawn.

 

Any comments/experiences/horror stories are welcome- and thanks!

               
               

               
            

[Legacy_Lord Sullivan]

               

While I have not dabbled in tileset creation in some time, I would have to say that the most issues I experienced that made FPS drop was mostly from other things then tilesets. But strictly speaking of tilesets, what I found to be the culprits in fps drop mostly was...

 

*Bad model (i.e. forgotten unneeded objects forcing the engine to create shadows for these extra objects when using shadows)

*broken/bad walkmesh (Not in all cases but, I have experienced it).

*Certain tiles with animated objects(again not all cases, but I never figured out why in the few instances/experiences. I did suspect some textures with transparency part of the animated object though but inconclusive)

*I can't think of anything else at the moment.


P.S. Separated objects using the same texture is not really an impact. Management of these objects to be grouped with others is mostly important when using shadows. Sometimes the opposite is true, you must break a single objects in pieces for properly working shadows.


I mention shadows allot because I always use them and I prefer for others to have the option to use them. So it is an ultimate necessity for models to have proper working shadows when I create models.

               
               

               
            

[Legacy_OldTimeRadio]

               

Agreed, but I'm mostly looking for some less straight-forward things that might slip into a tileset and cause FPS problems.  Even when the creator is trying to be careful to do a good job.  Things that traditionally might not stand out as causing lag but do or do when (whatever) is repeated a lot.  Like chimney emitters, for instance.  Individually, not a big deal but in larger numbers a simple intensive subsetting on the emitter (that you might not even notice if it was turned off) could cause a lot of lag.

 

The four things I listed were the only things I couldn't cross off my list of what might be causing a particular spot in a particular tileset to lag.  The idea being if I can suss out why that one spot was laggy, maybe there's something greater to be learned.  Every tileset has its laggy bits, of course.  This is the one I just happened to want to dig into. 

               
               

               
            

[Legacy_MerricksDad]

               

In my experience, your initial list would seem like this to me:

 

*a bunch of visibility nodes which make a very open map, instead of a realistic map -- On my machine, path finding is very fast, and relies on a much more complex algorithm than simple visibility pathing does, so I think this one is pretty much going to be a zero-engine-hit concept

 

*separate meshes using same texture -- In my experience, the texture is loaded only when the mesh is first seen, so the hit would be on loading only, after that, it would depend on your memory capabilities, and should not affect the FPS unless your machine is a calculator from 1996

 

*multiple tile lights (even when off) -- I can see this causing a FPS hit, as this is a calculation that it has to do every time you draw a frame. This also affects how your character's point of view lighting works, so as you move, it has to constantly detect and set that fading from lit-in-darkness to dark-in-lit-areas, which functions horribly on this machine (instead jerking from one to the other without a transition time).

 

*tiles with un-ordered nodes -- I'm not sure that would really be effective unless there were also A-nodes. The engine has to draw the same quantity, and without A-node capturing, wouldn't x+y simply be the same as y+x? Or do they have so much jammed in the render loop, that the loop itself could cause a FPS hit. If that is the case, then it would seem that nodes that WERE parented to something other than the aurorabase would actually take longer (loop inside loop). But think about this: Objects parented to nothing require a different method of shadow calculation from what I can see. Objects given a hierarchy know onto what they should be casting shadows and what they should not, more readily. The same with anything in the render loop looking for mesh opacity, or texture alpha. In these cases, order makes a big difference, and I could definitely see this taking some calculation time.

 

 

So here are the things I noticed when building Granitelands:

 

• Grass density: I realize this is obvious, because more needs to be drawn, but consider the fact that objects pop in and out of existence constantly. Creation and cleaning of objects is a major slow down on the processor. If done inside a render loop, any object that makes or destroys other objects on the fly will have a similar problem. Emitter planes take far less time to clean up, because I think they're re-used. On the other hand, emitters which drop a 3D object take forever to clean up. Emitters which drop emitters make the machine very angry. So anything that grows trees and then hacks them down has to traverse them a lot of extra times, not including erasure calls. This could include shadow poly faces.


• Geometry and shadows: As we know, the more complex a shadow, the more FPS hit. Obviously. But, think about this: A low complexity shadow still can be complexified by the target shape. This means that a tile rotated 90 degrees against the area sun's normal vector may have a very different shadow than one not rotated. It also means that two non-rotated tiles casting the same shadow will be affected by the adjacent tile's surface shape differently. That's a lot of math. So generally speaking, low poly shadows does not completely free the engine of pain if the target surface is high poly. But it is still better than high poly shadow with high poly target surface. Now add to that the lighting aspects (shadow cancellation), and the loop is even longer. So try rotating a few tiles in different directions, forcing a richer shadow terrain, and see what you get in FPS change.


• Concavity: Shadow math calculation lead me to complexity issues surrounding concavity. Any time there is concavity, this causes additional repetition if the concave surface axis is not completely perpendicular to the solar normal vector. In less technical terms, if the sun has to cast a shadow on to the object which causes the shadow, there is far more math. Let's call it self-shadowing. It seems the draw loop has to calculate, per face, all this +1 to -1 back and forth over and over again. It expects to NEVER find an instance on a single mesh where it will flip from +1 to -1 and then back again, so you will see shadow artifacts primarily on surfaces which are concave, and where the opening of the concavity is not pointed directly at the sun. Small concave sections on the same mesh are not even going to show a shadow artifact most of the time, but they still cause FPS hits because of this error. I found while doing all the shadow doctor scripts, that there MUST be a fault in the engine surrounding these concavities, because errors resulted in math that could only come from function exit, function failure, or divide-by-zero catch-and-ignore. I have not poked around with it yet, but I have to assume that deep concavity on meshes will be affected at least somewhat similar by non-solar lighting, so even with env-shadows disabled, there could potentially be a math-error farm surrounding shadows. Try turning shadows entirely off and see what happens in the same map. OC meshes primarily got rid of most of these errors by removing any downward facing planes. With an unlocked camera, we don't have that luxury.


• Shading: Like shadows, I think shading, which I don't think can be turned off, can be affected by concavity. I've never seen a shading error, but that doesn't mean there isn't any. Like with unseen shadow errors in concave meshes, you might be able to detect them on the same meshes if you watch FPS readouts. As I said, I've never seen an artifact drawn from a shading error, only shadow errors.

               
               

               
            

[Legacy_Zwerkules]

               In my Medieval City tilesets most buildings have a smoke emitter and some tiles have more than one smoke emitter and though they can be turned off, I never had to because the areas with lots of those smoke emitters still render fast enough.
What is causing the worst lag in my experience are extra lights on tiles (not the main or source lights) and a lot of meshes which use textures with an alpha layer.
               
               

               
            

[Legacy_Bannor Bloodfist]

               

From my experience, the ordering of objects in the mdl file can make a huge difference, and the more sub-sub-sub objects (ie take a close look at how badly things were linked in some_ux's ships.  They break very easily due to all of the objects being linked to so many parents... meaning object z was a child of object y, which was a child of object x (ad infinitum) and all finally linked to the alpha node...

 

So parent child relationships can add bad side-effects in many ways.

 

I also found that the direct order of objects in the mdl also affects things... meaning start with a flat plain that is the ground, add a wall for a building, but only the front facing wall, then add the roof, then go back and add the side walls... the order that they load makes a difference. 

 

Certainly, especially with more powerful cpus etc, the individual mdl may not cause a human noticeable issue, but things ALWAYS add up. Remember that Aurora only uses a SINGLE CPU, no matter how many you have in the box.  IE having a quad core processor really does not help with Aurora, and if I remember correctly, even after setting which core should be used, beyond that, additional cores don't help... except that your OS always resides on core 0 (or 1 if you count that way) you should set the core value for Aurora to use the 2nd core to give as much power as you can to the engine but the 3rd. 4th ad-infinitum do NOT help you at all.  So most you can gain is full power from Core 2, 3 or whichever you choose. 

 

Anyway, the main point I am attempting to make, any order issues do add up, sure each hit may be slight, but every hit you can avoid you should.  Back to what I had noticed with tiles, when objects were not ordered correctly, the processing hit(s) added up.  I successfully modified individual mdls in such a way as to make things almost random in when they were loaded by the engine, and even after compiling the mdls (which DOES make a difference) the order of the objects being drawn made a difference.  Again, small, but each hit counts.

 

The most problematic where objects attached to the alpha node, and when that alpha node itself is initially drawn.  It should always be last, other than sub objects... so, static objects should ALL be loaded before you get to all objects affected by alpha status.

 

Most tile-sets are multiple copies of whatever the first dozen or so tiles were created.  Meaning bits were typically copy-pasted into each new tile that was created.  A huge number of these copy-paste operations totally screwed up load order of the individual objects.  Again, small hits each time, but the more hits you get, the worse the lag gets.

 

Again, any ALPHA issues amplify the hits you get.  The more alpha surfaces you have, the more lag.  That doesn't mean you need to eliminate alpha, just that you need to pay attention to the individual objects load order.  Compiling can almost eliminate the lag, but not quite, as I mentioned I deliberately screwed up the order of object in a tile that had alpha textures applied and also had ani-mesh objects... by adjusting the order to more logical fashion, it did cut down on the lag.

 

As mentioned above, shadows cause issues, as we all well know.  The simpler the shape of a shadow casting object, the easier it is for the engine to handle all the math as MerricksDad mentions. 

 

Does it make sense to go through a huge tile-set and attempt to clean up these issues.... only the alpha issues are truly important, including shadow issues, but correcting any object attached to the alpha node can make a huge difference.  I would not attempt to go back through a large tileset, most especially TNO sized etc, but if I were already editing a given tile-set, most specifically editing the tiles via 3ds or gmax,, then by habit I also adjust the order things.  However, the issue with Gmax and most especially with 3dsmax, is that max remembers the object by it's own internal number and can cause the exported mdl to become screwed up despite your efforts.  I have been forced to do the clean up as best I can while inside max, export the mdl, and manually, using a text editor, re-order the objects, then once that mdl is re-imported, max will remember the NEW order and keep it etc... IE, this is not a single fix type of issue, as you edit and add/delete objects, the order it is used by the engine changes, and max does not keep things the way you want them. so you may have to do a final text edit, re-import and a final export, to force things into their respective proper order.

 

Believe me, I have skipped that step numerous times due to pressure/time constraints, and most times, the hits have been small, but whenever I had the luxury to re-order things, I would. 

 

Remember, the engine attempts to compile on the fly, and the draw calls necessary are affected by everything of course.  But say you have already drawn on screen, the entire building, and have shadows turned on etc, the engine attempts to calc what objects will generate a shadow and what objects will be affected by that shadow, mathematically if those objects are all ordered, it takes less effort to get the work done.

 

Other issues have been discussed in the past, and all, even the minor ones, do add to the overall lag buildup.  The most egregious are almost always related to texture and shadow - meaning alpha related.  Bad woks, too many, and wasted faces in woks, too many and wasted faces in objects etc.  Bad visibility maps, bad path-node maps add to the general complexity, and should be addressed at time of creation.  As mentioned way above, having every tile set with a bad "A" path-node and no specific visibility node selected cause various issues throughout the game play, from loading times to running times...

 

So, ALWAYS check alpha related issues... emitters, moving objects or animesh objects.     

 

You briefly mentioned how the engine reacts to objects and sometimes coughs up objects in unexpected order etc... and yes, you are right, the engine naturally expects things to be ordered in a specific way, but WILL handle most of what you can throw at it.  But why make it work harder than it needs too? 

 

We also noticed how groups react differently, and were never able to directly figure out why.  I think that the engine does an extra in memory compile on groups... and orders things how IT wants them ordered almost regardless of how the models were exported.  And the engine pops the entire group onto the screen in as few draw calls as possible.

 

As to lighting, yeah, lighting effects everything as has been mentioned multiple times through this thread and others.  More lights add to the math required to figure out not just shadows, but brightness, AND color rendition, all of which affect the time it takes to generate the scene.

 

Edit:  Forgot to actually ANSWER the initial question!

Sooo, finally, which affects load times the most?

 

In reverse order, I would set it as Lights have the most impact as they affect EVERYTHING, then node order, then it would be a toss-up for the last place between visibility node and not combining meshes.

               
               

               

                     Modifié par Bannor Bloodfist, 18 février 2016 - 07:50 .
                     
                  

            

[Legacy_OldTimeRadio]

               

Thanks, Bannor, and everyone else so far.  Each one of y'all have mentioned at least one thing that led to something interesting so thank you for taking the time to help.

 

Random observations:

 

Proceduretype TXIs: One of the things that was causing me a problem appears to be really poor performance of the proceduretypes used for water.  I did the tests last week and the typical water TXI's had a hit of 10-20 FPS on my 60FPS system.  Never did the bean-counting on that one.  That's a lot of FPS.  General rule so far seems to be anything except proceduretype cycle (which is basically golden) is potentially dodgy.  Wonder how many other people that affects?  Using the "DebugMode 1" and "trace fps" console commands (or FRAPS, which is much easier to read) and turning off texture animation in the advanced video options on an area is a quick way to see how much of an impact those're having on your system. 

 

So far, everything I've read seems to point to a proceduretype arturo, for instance, being done on the GPU.  But everything I'm seeing in the OpenGL debugger seems to imply it's actually done in the CPU.  At least on my machine.   

 

Tiles/groups and how they're drawn: Lots of looking at how groups are drawn versus tiles (using AMD's CodeXL) and even though I am using between 3-5 different scenes on a regular basis, I still can't really say much that's definitive.  It looks like the engine tries to draw ground planes in a wave out to max view distance from the player and then another wave of geometry from max view distance back in towards the player.  This is a bit oversimplified but that seems to be the idea, anyway.  In practice, not so much.  In a perfect world, it also seems to try to draw tile lights first and all a-node stuff last.   This, also, is not guaranteed to happen properly and both Bioware tilesets and community ones seem to have about the same hit-or-miss ratio.

 

Interestingly, from running some of these scenes several different times, one draw call at a time, it is using some kind of logic and is consistent, which is good.  I could, for instance, stand in a little 2x2 area (of individual tiles) and see the ground being drawn out and the tile clutter being drawn back towards me nomatter which corner of the area I stood in.

 

I am operating under the hypothesis that drawing all the parts of the visible tiles in a scene haphazardly (including alpha stuff out of order) is "bad".  Maybe this is accurate, maybe not.  My 16x16 rip of Megaton as a tile group and Thrikreen's 10x10 (?) arena tile group draw very "good".  My observations of tile groups' performance (and my gut 'feeling') seems to jive with Bannor's point about tile groups getting a little extra love from the engine...but many smaller Bioware tile groups don't seem to behave as well as one would expect.  There may be some extra love from the engine for tile groups, but I'm also starting to suspect that pivot point placement (or possibly bounding box) may also play a role.  As in, the engine has an idea of how it wants to draw things and if the pivot points are placed in such a way as to help that process (by being at 0,0,0), then it might be able to do a better job.  Obviously, pivot point at 0,0,0 might be problematic for animated meshes but maybe not for static ones.  Just a thought.  Or, if that doesn't pan out, attaching non-a node meshes to the ground plane as children (and maybe even a chain of them, one attached to another) might force the whole chain of them to be drawn in sequence and, in that way, also facilitate the engine getting things on the screen most efficiently.  Many tiles I've looked at are mostly "flat", where many individual meshes are attached directly to the base.

 

Theoretically, any of the above should be testable by duplicating an existing group and "optimizing it" and then be able to compare FPS between that and the existing group or by producing a ground tile (because it's easy) which is overstuffed with geometry and/or alpha and be able to show that that has a better or comparable FPS than an average number of default Bioware tiles.  Something like that.

 

Also worth pointing out that whatever logic is involved, it was probably never optimized for an unlocked camera/low camera angles.

 

Node hierarchy:  The internal numbering that Bannor describes in his post is the "part" number, I believe.  This will show up in compiled models and NWN Explorer Reborn will show you this for them and calculate it in uncompiled models.  AFAIK, it's the "true" order of the model nodes and, as Bannor said in the 10th paragraph in his post, can be modified by copying around node blocks in notepad.  Anecdotally, I have read that, yeah, Max has some funny ways of ordering nodes.  From the bit I read it was something like most recently edited parts wound up last in a given node order.  BTW, you know all those threads where somebody is trying to get soemthing alpha based to show up correctly, and it doesn't and they have to fiddle with it forever to get it right?  This might be a big part of that.  

 

For people using VelTools (and I can't recommend it enough), Toolbox III has a utility which allows you to print the true hierarchy to the MaxScript listener and to reorder the hierarchy by mesh name.

               
               

               
            

[Legacy_MerricksDad]

               

You think arturo is bad, try the other three deformations. It gets nasty.

 

Another thing about the node ordering is the difference between arrays and objects or class objects. An array knows to store things from 0 to n, where the objects don't actually retain a proper order at every request. I learned this when playing around with libraries I was forcing vb10 and C# to work together better. So what you may see in the tree structure readout given by max, when dumped, will be in the order supplied either A) given by their order in memory or some bucket, or their order given the branch to sub-branch system (which may mimic an array). When you examine them under "order children" on the nwmax panel, what you see is an array underlying the tree structure. For more information, you can research the difference between arrays and lists in C# and VB10. While a list can be made to report elements given an index, the default method is sloppy and does not rely on indices, rather than keys, and does not even require indices be given. Indices are just a secondary attribute. An array is always by index, and generally speaking, only by index. Most C-built tree structures are pre-written external to the software you use, and commonly require nothing but list classes.

 

In lay mans terms, if you ask a parent what it's first child is, it may forget the name of its first child and give the dog's name first. But if you ask again, it may correct itself and give the name of at least one of its human children. Though you will never be sure which answer is correct without an actual index system or underlying array.

 

So this brings me to a realization that draw order may show some inconsistency or unreasonable behavior if they don't use an actual array. It may end up calling a random child from a bucket. I'm more tempted to think that once the engine realizes the model structure, that the children are kept in arrays than lists if you see a strict order over like 100 draws of the same scene.

               
               

               
            

[Legacy_Asymmetric]

               

Yeah, tell me about it. The node ordering seems completely arbitrary. That causes issues with supermodels, i.e. when the child has a different ordering than the parent.

 

I learned that the hard way while writing neverblender. The panther's animations were completely broken after importing and exporting again. The only difference to the original was the order of the nodes. (Blender gives me objects in alphabetical order)

To fix it I had to save the order of the nodes during import and restore it for export.

               
               

               
            

[Legacy_MerricksDad]

               

I have no issues at all if I use the nwmax "order children". I think it does exactly what it was meant to. Has anybody else had issues with that feature?

               
               

               
            

[Legacy_OldTimeRadio]

               

A particular/similar vantage always draws tile meshes in the same order, whatever that winds up being.  I rely on this to help make sure I don't accidentally skip over the OpenGL API calls for a mesh I'm investigating.  

 

As far as the NWMax "order children", I totally forgot that was even in there.  I tend to use VelTools a lot more than NWMax for some things.  Having said that, I just set up a simple hierarchy, confirmed the order with VelTools, changed the order with the NWMax "order children", exported it, checked the text file and the nodes were in the right order, compiled it with the Bioware compiler and then used NWN Explorer reborn to confirm the internal "part" numbers reflected the change.

 

So, seems legit and I like the interface more than the one VelTools uses.  Thanks for bringing it up!

               
               

               
            

[Legacy_Asymmetric]

               

Oops, I think I expressed that wrong. I meant when the nodes in a model are ordered differently than the nodes in its supermodel. At least it caused issues with animations during my tests. After saving and restoring the original node ordering everything worked fine.