Author Topic: Fomenting Mutiny  (Read 6817 times)

Legacy_OldTimeRadio

  • Hero Member
  • *****
  • Posts: 2307
  • Karma: +0/-0
Fomenting Mutiny
« Reply #225 on: January 27, 2014, 11:25:59 pm »


               Here is a simple scenario that illustrates why taking any model or batch of models from CEP (or anywhere else) and just running them through CleanModels is going to risk being a boondoggle.  This is not, in any way, meant to undermine the usefulness of CleanModels.  It can be a very useful tool, but it's not necessarily useful in all situations and, frankly, the results from CleanModels can be very misleading in evaluating the quality of a model. 

This misleading evaluation thing?  It's not CleanModels or OldMansBeard's fault.  This is a perceptual flaw on the part of many of its users.  It's a very easy mistake to make.

1. Make  simple teapot placeable.  Here's how I did that, and the rest on this line is only important if you want to reproduce my work: Make an aurorabase.  Make a teapot in GMax (Radius:250, Segments 4).  Move to X, Y 0.0, Z 250.0.  Drag and drop a texture on it.  Convert the teapot to editable mesh.  Add an Aurora Trimesh modifer with whatever settings you like.  I use Render and Override Material Values only.    Attach it to the model base and export.  I export geometry only, in this case. 
2. Run it through CleanModels.  Outside of the default settings, I'm doing a binary snap because this model is headed for the compiler when it's done.
Result:
plc_a01.mdl loaded.
  binary snap applied to node positions and mesh vertices
  vertices welded in [teapot01]
  null faces deleted from [teapot01]
  unused tverts deleted from [teapot01]
  tverts welded in [teapot01]
  renormalized tverts in [teapot01]
  Fixes made = 11073
plc_a01.mdl written.

Total Fixes = 11073

Okay, 11,000 fixes.  Are there really 11,000 errors that need fixing in that simple placeable?  No, of course not.  Are there 11,000 operations that CleanModels has carried out against that model which were at variance with the settings it was run with?  Oh yes, I have little doubt of that. 

However, 11,000 CM3 fixes ≠ 11,000 actual errors of signifigance.

Are some of those errors signifigant?  I'm sure, in some way, they are or could be argued to be.  Are those errors noticable in-game and/or do they have a negative impact on the Aurora engine?  That is more up for debate, IMO.  But that's far from the point I'm trying to make, which can only be evinced by compiling these models and having CleanModels analyze them again.

Before we do that, though, let's take the outputed model from CM3 and run it through CM3 again, with the same settings.  Looks like there are still some errors left, according to CM3:
plc_a01.mdl loaded.
  binary snap applied to node positions and mesh vertices
  renormalized tverts in [teapot01]
  Fixes made = 444
plc_a01.mdl written.

Total Fixes = 444

Again are there really 444 errors of signifigance in this model?  No, there aren't.  You can keep taking the outputted model and feeding it back into CM3 with the same settings and you're going to get a notification that there were 444 fixes performed each time.

Even all this so far, as I said, is not my main point.  Let's take that model, that CleanModels produced, and compile it with various compilers and then have CleanModels analyze those binarized models:

Internal BioWare compiler results:
Total Fixes = 9271
DLA Model Compiler: Total Fixes = 9271
Acaos mdlcomp from Explorer Reborn 1.63: Total Fixes = 9271

After compilation, all three are reported by CleanModels to have gone up 20x the number of errors that needed to be fixed.  Interestingly, all by the same amount. 

What's actually going on here?  What does all this mean?

It means: If you load a model into CleanModels on the hope of looking for errors, you will likely not be disappointed.  You will likely see hundreds if not thousands of errors, just like you expected.  Using those numbers to support any kind of assessment of the quality of the model is often specious.  Only a relatively skilled modeler can actually make that assessment, especially with complex geometry.  The teapot placeable I made compiled just fine with the Bioware model compiler and worked fine in-game.

With the NWN2 models, and with basically anything else in the existing CEP package, I think the main questions should be:
* Is this model reported to have a specific error of some sort in the first place?

If so:
* Does the model compile well?
* Does it look good in game?
* Is there anything about it that taxes the engine, unnecessarily, and needs to be optimized?

CM3 is a powerful and useful tool, but feeding models into CM3 can lead to unintentional trips down the rabbit hole if one isn't careful.  Worse, its output can be misinterpreted to call into question the quality of models which are, at least in this case, more or less perfectly fine.
               
               

               


                     Modifié par OldTimeRadio, 27 janvier 2014 - 11:27 .
                     
                  


            

Legacy_The Amethyst Dragon

  • Hero Member
  • *****
  • Posts: 2981
  • Karma: +0/-0
Fomenting Mutiny
« Reply #226 on: January 28, 2014, 12:03:56 am »


               Thanks, OldTimeRadio.  Very insightful.

I was using it mainly because of reported ambient/diffuse settings being too dark on some of the models.  It also welded some vertices for me. '<img'>
               
               

               
            

Legacy_omen_shepperd

  • Sr. Member
  • ****
  • Posts: 347
  • Karma: +0/-0
Fomenting Mutiny
« Reply #227 on: January 28, 2014, 01:33:56 am »


                I read back on page 1 or 2 where you could use some
blueprint builders to help make ERF’s. That is something I can do and would be
very willing to help out with. I wanted to know if anyone else is making blueprints
as well so I did not make any that conflicted with ones already made. 


I will join the Wiki page and hopefully I can help out in other ways while I am
learning Hak work and scripting. 
               
               

               
            

Legacy_FunkySwerve

  • Hero Member
  • *****
  • Posts: 2325
  • Karma: +0/-0
Fomenting Mutiny
« Reply #228 on: January 28, 2014, 02:27:23 am »


               

omen_shepperd wrote...

 I read back on page 1 or 2 where you could use some
blueprint builders to help make ERF’s. That is something I can do and would be
very willing to help out with. I wanted to know if anyone else is making blueprints
as well so I did not make any that conflicted with ones already made. 


I will join the Wiki page and hopefully I can help out in other ways while I am
learning Hak work and scripting. 


Ideally, blueprints would be generated automatically by a perl script. That's what acaos did for all the places we made for 2.3. Much less room for pebkac.

Funky
               
               

               
            

Legacy_FunkySwerve

  • Hero Member
  • *****
  • Posts: 2325
  • Karma: +0/-0
Fomenting Mutiny
« Reply #229 on: January 28, 2014, 02:30:08 am »


               And here's the script:


//
// --------------------------------------------------------------------------------
// This script generates static placeables from placeables.2da.
// --------------------------------------------------------------------------------
//
//   #!/usr/bin/perl -w --
//   #
//
//   use strict;
//
//   use NWN::Gff;
//   use NWN::GffRead;
//   use NWN::GffWrite;
//
//   while (defined(my $line = <>)) {
//     my ($app, $static, $name) = split(/\\s+/, $line, 3);
//
//     next if ($name =~ /\\*\\*\\*\\*/);
//     $name =~ s/"//g;
//     $name =~ s/\\s*$//s;
//     $name =~ s/\\s*\\([A-Za-z0-9]+\\)\\*$//;
//     $name =~ s/\\s*\\*$//;
//
//     my $res = sprintf("plc_static_%05d", $app);
//     my $utp = undef;
//
//     if (-f "temp0/$res.utp") {
//       $utp = NWN::GffRead::read('filename' => "temp0/$res.utp");
//     } else {
//       $utp = NWN::GffRead::read('filename' => "temp0/plc_static_00001.utp");
//
//       $utp->{'Tag'}            = $res;
//       $utp->{'TemplateResRef'} = $res;
//       $utp->{'PaletteID'}      = 4;
//     }
//
//     if ($name =~ /VFX: Flame/) {
//       $utp->{'PaletteID'} = 231;
//     } elsif ($name =~ /Lightshaft/ || $name =~ /Shaft of Light/) {
//       $utp->{'PaletteID'} = 230;
//     } elsif ($name =~ /Portal [0-9]/) {
//       $utp->{'PaletteID'} = 232;
//     }
//
//     $utp->{'Appearance'} = $app;
//     $utp->{'Static'}     = ($static == 1);
//     $utp->{'LocName'}    = { '0' => $name };
//
//     NWN::GffWrite::write($utp, 'filename' => "temp0/$res.utp");
//   }
//
// --------------------------------------------------------------------------------
//
//


Funky
               
               

               
            

Legacy_FunkySwerve

  • Hero Member
  • *****
  • Posts: 2325
  • Karma: +0/-0
Fomenting Mutiny
« Reply #230 on: January 28, 2014, 02:31:54 am »


               Er, sorry for spam posting. Code is acaos'. It's also still in nwscript-commented format, as that's taken from our mod, and we didn't want it throwing compiler errors.

Funky
               
               

               
            

Legacy_Pstemarie

  • Hero Member
  • *****
  • Posts: 4368
  • Karma: +0/-0
Fomenting Mutiny
« Reply #231 on: January 28, 2014, 12:13:51 pm »


               

OldTimeRadio wrote...

...CM3 is a powerful and useful tool, but feeding models into CM3 can lead to unintentional trips down the rabbit hole if one isn't careful.  Worse, its output can be misinterpreted to call into question the quality of models which are, at least in this case, more or less perfectly fine.


This is exactly why I use CM3 primarily for tiles and to fix specific errors or perform specific tasks. As OTR points out CM3 is VERY good at what it does, but it is not perfect (although its pretty close). In the example OTR used to prove his point, I have little doubt that on the second trip through CM3, those 444 errors were 444 operations CM3 performed becasue the settings FORCED it to, not because it HAD to apply those fixes.

All this being said, I would like to add that CM3 was designed for tiles and was not originally intended to work with other model classes. When it was discovered that it COULD work with other model classes, OMB added the parameters to allow it to do so. What this means to me: if you run a model through CM3 the first time and CM3 says it has 10,000 errors, that model had 10,000 errors. Simply put - CM3 doesn't lie, but it can get confused if you keep running a model through it, and, quite frankly, if CM3 does keep finding errors, you might want to look at the model and what CM3 might be trying to tell you.

':whistle:'
               
               

               


                     Modifié par Pstemarie, 28 janvier 2014 - 01:06 .
                     
                  


            

Legacy_Zarathustra217

  • Sr. Member
  • ****
  • Posts: 322
  • Karma: +0/-0
Fomenting Mutiny
« Reply #232 on: January 28, 2014, 12:29:15 pm »


               Regarding CM3, the main cause of "errors" is simply a matter of how you decide to round all floats (the "snap" option). The best method here really is binary, at least for the final product.

I'd also recommend disabling the "Allow Splitting" option when doing mass operations. Most shadow issues (at least the significant ones) are already solved by welding and re-pivoting.

Anyway, I'd like to reiterate the concept of making overhauls of some of the current CEP content (as I suggested above) - any scepticism to that?
               
               

               


                     Modifié par Zarathustra217, 28 janvier 2014 - 12:29 .
                     
                  


            

Legacy_Pstemarie

  • Hero Member
  • *****
  • Posts: 4368
  • Karma: +0/-0
Fomenting Mutiny
« Reply #233 on: January 28, 2014, 03:35:16 pm »


               

Zarathustra217 wrote...

Regarding CM3, the main cause of "errors" is simply a matter of how you decide to round all floats (the "snap" option). The best method here really is binary, at least for the final product.

I'd also recommend disabling the "Allow Splitting" option when doing mass operations. Most shadow issues (at least the significant ones) are already solved by welding and re-pivoting.

Anyway, I'd like to reiterate the concept of making overhauls of some of the current CEP content (as I suggested above) - any scepticism to that?


Grear points about CM3. Another thing that helps with the snap option is making sure that the texture size you're snapping to corresponds to the texture size you're using. Just a little tidbit OMB told me when I first started using CM3.

Overhauling models is definitely a good idea and an update - which is what TAD is working through now - offers the best medium through which to do so.  I'd say that, as TAD noted in an earlier post, if someone sees something that needs an overhaul, feel free to do it, and email the work to the address he provided.

The only problem at this point is tracking the stuff, but I see people are already exploring that as well. Tracking progress could be as simple as creating an excel sheet that lists all the CEP assets and authors can note what they are working on. However, I'm not sure if there is some utlilty out there that can port a ak file's content list to an excel sheet and how hard it would be to create one if TAD wanted to do the list idea.
               
               

               
            

Legacy_Tiberius_Morguhn

  • Sr. Member
  • ****
  • Posts: 396
  • Karma: +0/-0
Fomenting Mutiny
« Reply #234 on: January 28, 2014, 05:05:30 pm »


               One question - and it's something I've noticed on TAD's wiki - why are tilesets being proposed as additions to CEP?  I may be in the minority opinion on this, but I stopped at CEP 2.1 for my own projects because of the addition of tilesets and other overreaching content that didn't really fit.
               
               

               
            

Legacy_Zarathustra217

  • Sr. Member
  • ****
  • Posts: 322
  • Karma: +0/-0
Fomenting Mutiny
« Reply #235 on: January 28, 2014, 05:18:52 pm »


               

Tiberius_Morguhn wrote...

One question - and it's something I've noticed on TAD's wiki - why are tilesets being proposed as additions to CEP?  I may be in the minority opinion on this, but I stopped at CEP 2.1 for my own projects because of the addition of tilesets and other overreaching content that didn't really fit.


I somewhat agree, and it does add to the amount of work involved with maintaining the CEP. And after all, the CTP package was made to fill out that role. Perhaps they - along with the other addons - should be made into seperate downloads, and possibly only as a legacy thing. Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even).

Anyway, so far, I've been tracking down models that have improper lighting settings and fixing them manually in the files, but if we use conservative settings for CM3, perhaps we could allow ourselves to pass all models through it? If processing time is a worry, I've got a reasonably fast CPU. It would still require some testing, but I imagine a beta-phase before release could catch most errors (given they arise) if we get enough testers.

As for overhauling - perhaps we should add a new page to the wiki for suggestions of overhauls? (like the current for suggestion new content).
               
               

               
            

Legacy_MerricksDad

  • Hero Member
  • *****
  • Posts: 2105
  • Karma: +0/-0
Fomenting Mutiny
« Reply #236 on: January 28, 2014, 05:38:04 pm »


               

Pstemarie wrote...

However, I'm not sure if there is some utlilty out there that can port a ak file's content list to an excel sheet and how hard it would be to create one if TAD wanted to do the list idea.


I was just in the process of exploring HAK, ERF and MOD formats for use with my AuroraExplorer. I could easily create an exporter to a tab or space delimited file, such as CSV. Excel and OpenOffice should be able to open that just fine. This isn't top on my priority list, but it does need finished to have this explorer complete.
               
               

               
            

Legacy_Proleric

  • Hero Member
  • *****
  • Posts: 1750
  • Karma: +0/-0
Fomenting Mutiny
« Reply #237 on: January 28, 2014, 06:16:39 pm »


               Given that tilesets are already in scope for CEP 2.4, I've suggested some (hopefully non-controversial) fixes and additions on the wiki. However, for many reasons, it might be wise to defer consideration of the more recent tilesets for the time being.
               
               

               
            

Legacy_Proleric

  • Hero Member
  • *****
  • Posts: 1750
  • Karma: +0/-0
Fomenting Mutiny
« Reply #238 on: January 28, 2014, 06:31:04 pm »


               

Zarathustra217 wrote...

...Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even)...

A game of two halves.

I've offered in another thread to share my version of the old CEP horses that works with the 1.69 phenos and horse scripts (subject to agreement on whether scripting is in scope for future releases). That would make the CEP mount phenos redundant, going forward.

However, those phenos would have to be retained for backward compatibility; at most, we could freeze further development. Likewise, there are other phenos which remain useful, such as swimming/flying, and of course the Bioware-compatible phenos for skeletons and custom races.
               
               

               
            

Legacy_The Amethyst Dragon

  • Hero Member
  • *****
  • Posts: 2981
  • Karma: +0/-0
Fomenting Mutiny
« Reply #239 on: January 28, 2014, 07:24:03 pm »


               

Zarathustra217 wrote...

Tiberius_Morguhn wrote...

One question - and it's something I've noticed on TAD's wiki - why are tilesets being proposed as additions to CEP? I may be in the minority opinion on this, but I stopped at CEP 2.1 for my own projects because of the addition of tilesets and other overreaching content that didn't really fit.

I somewhat agree, and it does add to the amount of work involved with maintaining the CEP. And after all, the CTP package was made to fill out that role. Perhaps they - along with the other addons - should be made into seperate downloads, and possibly only as a legacy thing. Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even).

Anyway, so far, I've been tracking down models that have improper lighting settings and fixing them manually in the files, but if we use conservative settings for CM3, perhaps we could allow ourselves to pass all models through it? If processing time is a worry, I've got a reasonably fast CPU. It would still require some testing, but I imagine a beta-phase before release could catch most errors (given they arise) if we get enough testers.

As for overhauling - perhaps we should add a new page to the wiki for suggestions of overhauls? (like the current for suggestion new content).


Proleric1 wrote...

Given that tilesets are already in scope for CEP 2.4, I've suggested some (hopefully non-controversial) fixes and additions on the wiki. However, for many reasons, it might be wise to defer consideration of the more recent tilesets for the time being.

As far as tilesets go, I think the community has made a lot of better ones over the years.  Luckily, the ones already included in the CEP are optional for builders to use.  Fixes for issues some of them have would always be welcome.

Proleric1 wrote...

Zarathustra217 wrote...

...Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even)...

I've offered in another thread to share my version of the old CEP horses that works with the 1.69 phenos and horse scripts (subject to agreement on whether scripting is in scope for future releases). That would make the CEP mount phenos redundant, going forward.

However, those phenos would have to be retained for backward compatibility; at most, we could freeze further development. Likewise, there are other phenos which remain useful, such as swimming/flying, and of course the Bioware-compatible phenos for skeletons and custom races.

I'm going to be sorting through the CEP's pheno1 to pheno5 haks to sort out which files belong to "normal" (phenotype 0-8) brownies and wemics, if any, and move them to the core haks.  I'd do the same looking for any robes with phenotypes 0-8 and pull them from the pheno haks, moving them into the core haks.

This should leave it so that the pheno1-pheno5 haks can be completely optional.

Proleric1: If you want to contribute a version of the old CEP horses that works with the new system, I think that would work out great, and would keep those horses from being completely outdated.