Animesh doesn't necessarily break when compiled. I compiled the CTP Babylon tiles with animesh "wavy water" like TNO and they worked fine. But it might depend on how the model is constructed and how the animations are defined.
Bioware had their own model compiler, different from nwnmdlcomp. It was available for download at one time and I still have a copy of it. It was basically the same code as the engine uses to load ascii models, but pulled out into an executable, The compiled models it generates are slightly different from the ones nwnmdlcomp generates but the differences are not usually important unless you are doing tricky stuff with emitters.. (edit - added: it was never updated to corespond to patch 1.69, which may be why the tno models were left uncompiled)
Some Bioware tile groups have meshes that project across tile boundaries, they are compiled and they don't cause SQRT Domain errors in the toolset. Maybe there's more to it, than just that?
Come to think of it, as an extreme case, for the tests I was doing in the "draw calls" thread, I made some tiles that were 16 times bigger than normal, compiled them and they didn't break the toolset. So compiling tiles with overhangs as big as 155.00 metres definitely isn't a problem.
What does CM3 say about your test models? If it passes them and they break when compiled, then CM3 needs fixing because it shouldn't.