Merricksdad's Black Hills Tileset (First Look)

[Legacy_ShadowM]

               

^ no pivots moved, but whole mesh offset instead

 

I think nwnmax auto centers the pivot or mesh, I noticed it does with placables at least. I not sure if that related to tilesets. So you have to center pivot to keep the mesh from moving.

               
               

               
            

[Legacy_MerricksDad]

               

In gmax, if you change the pivot, the mesh origin is unchanged. If you change the mesh origin, the pivot automatically adjusts to compensate, staying away from the new origin equal to the set pivot point3 value. NwMax 0.8b makes a mistake on tile export and requires pivot to be [0,0,0] but fails to check if the coordsys is currently in world or parent. It should actually be setting the coordsys to the aurorabase, and then checking the pivot value, or otherwise doing the math to get the same result. I changed that mistake in my own version of NwMax 0.8b, fixing the hideous tile export issues.

 

When I move the shadow mesh sections an offset, I am actually moving the mesh origin, which automatically moves the pivot. Since I keep the pivot equal to the origin, it then matches the origin after a change. I then move the faces back to where they were by shifting them equal to negative the mesh origin change. This puts the pivot easily behind all faces on that mesh, but makes the mesh origin non-zero in relation to the aurorabase, which might be a problem.

 

If that is a problem, it should have showed up when I changed the mesh origin/pivot to the face centers in the trial where I explode the faces into separate meshes, but it does not show up there. I find it strange that separating individual faces out and perfecting their pivots does work, but that the work and memory load is the highest, and most unsavory technique.


Passing the split walkmesh, or split high poly mesh, through the NwMax 0.8b shadow checker gives no errors, meaning the pivot is within the positional range it needs to be (see code below from NwMax script: aurora_gui_speedbtns.ms)

   Spoiler
   

            face = getface obj ii
            normal = getfacenormal obj ii
            vertex = getvert obj face.x
    
            dp = dot normal vertex
    
            if ((dot normal obj.pivot) - dp > 5e-3) then ( errors += 1 )

 

So there must be other aspects to shadows which cause shadow errors. I'm personally seeing shadow holes, shadow spikes, negative shadow vs placeables, and negative shadow dot-dot-dot lines leading down slope faces. I don't see any of that with the method which explodes faces to separate meshes. So I wonder what is in the math that is crashing the shadow engine? It isn't simple complexity by facecount, because my low poly walkmeshes should have worked just fine, verts un-welded or otherwise.

 

I think at this point, if I can make an automatic shadow creating script, it will just be exploding all faces on a mesh to separate meshes and then setting the submesh origin to the face center. It works, but it gives Gmax a stroke more often than not, so I'm not too thrilled with the idea. Also, there is no way I can make high poly shadows with that method. Instead I have to use the walkmesh quality or lower.

 

I would like to next try splitting the walkmesh into small face groups, and remainder single faces. If I can take the intersection of two to four faces' negative normals, I can find the exact 0-value offset for that face group and adjust the pivot accordingly. I suspect most groups will only allow 2 faces, because you can ALWAYS find a shared negative normal position which matches both faces, as long as there is not a shared positive normal position, and the face normals are not exactly the same. The chances go down that such a negative normal exists for each face added to the group. I know the NwMax shadow checker uses 0.005 accuracy, but why not just head directly to 0.0?

               
               

               
            

[Legacy_MerricksDad]

               

a few hours of research shows that the only thing I can do other than exploding faces is to:

 

1. draw an imaginary line through every face normal

   
2. select any faces where lines come within such a distance from each other, or actually intersect (very rare), behind the faces

   
3. detach those faces to another mesh and set the pivot point to the midpoint of the distance between the points of nearest approach

   
4. hope for the best

While the math is fast, I personally don't have much faith in this approach, but it might be worth a try at a later date. For now, I will continue on with the manual labor of making a crap ton of shadow meshes.

               
               

               
            

[Legacy_s e n]

               

can't you:

-calculate the average normal for all the faces of a mesh

-get the vertex(s) within a certain distance from the normal line

-create the most simple mesh from the given vertexs and flatten that to mako your shadow mesh

               
               

               
            

[Legacy_MerricksDad]

               

If I fragment it first into the 6 prismatic regions, that would work about the same as doing the normal line matching I was mentioning.

 

I did a bit more testing last night before I passed out, and found that mostly flat shadow meshes with only a little bit of contour, such as the 0000 tile which has corners raised by 25cm, casts a shadow onto itself even if the shadow box is moved -5cm in relation to the aurorabase. That just blows my mind.

 

I also tested how the shadow is constructed on tile 0002, and found that of all the tiles, that tile was the most likely to create a proper shadow. Similar tiles for other height corners also worked well. But no matter how I cut the shadow box, even full exploding it, tiles such as 0022 would not render a shadow below the height midpoint of the mesh. This needs further looking into. I think it may have something to do with the difference in origin on those tiles which are offset in the z-axis to work with your multi elevation system.

 

 

The tile in the image above is 0022, with vertical offset forced to +1 height transition in the set file. It seems tile shadows are never drawn below the height transition base line. This is a major problem. To allow shadows to form, I may have to change the base height of every tile to be equal with the largest forced height offset in the whole set. I will be trying that today. It also helps explain why in some of the higher quality tilesets, that the base height of any tile is way above zero.

 

That discovery explains this hole in the sheet I have been trying to figure out for a day now. The point circled has a center which is up to 25 cm below the base line of the tile.

 

 

Another thing I noticed about tileset shadows is that they are permanent and cannot be erased by other light sources. They also seem to spawn from a point above the tile, but localized to the tile, in such a way that every duplicate tile, no matter where on the map, will show the same shadow shape without stretching for x/y position of the tile. That is very confusing to me why it would do that. Mostly the fact that it is not cancelled by light. Seriously, how does that work?

 

After a full tileset test, less than 1/5 of the tiles had a proper shadow, even when faces were fully exploded and perfect pivots were found for each face. In addition, there was often lines of light cast between two matching edges, as if the shadow casting math is making rounding errors. This points to a good reason to not break shadows into too many pieces, as it increases the amount of escaping light around mesh edges.

 

I am very tempted to exchange the shadow for light casting polygons and see where the origin comes from, or how the bad shadows are being drawn. I'd really like to see an edge drawn where the shadow is coming from. That would help me figure out the best system for making a shadow box builder.

               
               

               
            

[Legacy_OldTimeRadio]

               

Hey, if you aren't already using it as a resource, be sure to check out the archived CTP forums at HarvestMoonConsortium.  Threads like this can be pretty helpful.  

 

site:harvestmoonconsortium.com "shadows"

 

Great work so far, very impressive!  

 

BTW, if you're looking for the MaxScript Danmar mentioned in that thread, here it is.  

               
               

               
            

[Legacy_MerricksDad]

               

Reading that is showing exactly what I have found so far. I moved the tile meshes up 10m and almost everything was fixed, except pivot point issues. Especially at z=0. I still need to work on pivots, and I am also trying some stuff with full shadow boxes with faces representing a vertical wall at the unseen sides of meshes, where it goes underground. There is a seam at every mesh shadow and I suspect this extra box wall will get rid of it.

               
               

               
            

[Legacy_OldTimeRadio]

               

Crap!  Wait, wait!  I'm going to seriously apologize for that last message.  In attempting to be helpful I have unintentionally screwed up, bigtime.  I should have linked to this thread instead, highlighting Bannor's message.  Fixing shadows, especially on hilly terrain, has a bodycount associated with it.  I certainly won't try to dissuade you from doing whatever you like but just comment that the shading on your hills is already very, very nice- and that's before they've even been smoothed.

 

Part of the Modder's Hippocratic Oath is "I shall not knowingly point another modder down the path to insanity."

 

I blame no tea yet.   '>

               
               

               
            

[Legacy_Zwerkules]

               

I blame no tea yet.   '>

No tea yet? That's a horrible thought!

               
               

               
            

[Legacy_MerricksDad]

               

Lol

 

Crap!  Wait, wait!  I'm going to seriously apologize for that last message.  In attempting to be helpful I have unintentionally screwed up, bigtime.  I should have linked to this thread instead, highlighting Bannor's message.  Fixing shadows, especially on hilly terrain, has a bodycount associated with it.  I certainly won't try to dissuade you from doing whatever you like but just comment that the shading on your hills is already very, very nice- and that's before they've even been smoothed.

 

Part of the Modder's Hippocratic Oath is "I shall not knowingly point another modder down the path to insanity."

 

I blame no tea yet.   '>

 

While I like that thread, there seems to be a bit which I can actually work with.

 

So far, I have built these:

 

• yesterday's script to break a mesh into 6 directional sub meshes: up, down, left, right, north, and south. If a face normal points majority toward one of those directions (ie. y=>x and y=>z and y=>-x and y=>-z and y>=0) then it is considered pointing that direction and is put in the sub mesh of that name. Technically, there should NEVER be a down group, since I am using walkmesh-ready shapes, so if I do find one made, it should be deleted (as well as raising a question of walkmesh validity).


• new script to put dirt-block shadow casting faces down to Z=0, making it so shadows will be cast from the inside of the earth under the mountain, filling in see-through shadow seams at tile edges.


• updated script to NOT modify pivot points, because as far as I can see, modifying the pivot points is causing MORE issues, not less. To ensure the pivot for any given shadow mesh was behind ALL the faces on that mesh, I moved the pivot 500cm behind its current point (which was original model base center [posy, posx, model height offset+1000] ). This caused untold number of shadow artifacts, of all sizes and directions. Retaining the pivots at original model base center removed most of those artifacts.


• new script to examine shadow portions (those 6 directions of shadow) for double face cross rays. When a face normal ray from any face center, either forward or backward, hits another face on the same mesh, that face is flagged red (and will be separated into an Nth mesh later)

 

Right now I am mass running the new shadow doctor on the current shadow boxes. So far, it has found no errors, but it certainly might. I had to weld a few shadow blocks back together to find a mesh which threw the error message, just to test if the raytracer even worked in Gmax. Luckily, it does!

 

Hopefully I can find out why this happens, because as far as I can see, there is no ray going through any two faces on a single one of the 6 sub meshes, at least not from one face center.

 

Before I go too much further, could I get a yea or nay on who all has disabled shadows? This seems important.

 

Edit:

 

The shadow doctor does find a few faces which are offensive after the main mesh is fragmented into 6 portions. Now checking to see if separating those offending faces to another submesh fixes the issues. This could lead to about 10 total sub meshes per tile to create a good shadow casting box.

               
               

               
            

[Legacy_MerricksDad]

               

Something was said on one of the threads listed along the lines that the shadow is drawn from the pivot point. If that is the case, can the pivot point be used to denote where the sun is? Having the pivot point be the mesh center (which I assume is what most people do) seems like it would cause the shadow to cast in weird directions. I've always had the pivot point be the mesh [0,0,0], which just also happens to be the aurora base (until now).

 

I have not fully tested this concept, but I was able to make longer conforming shaped mountain shadows by placing the pivot point -10m below the mesh. I should be able to equally make the shadows shorter by moving the pivot up 10m from local 0. This I must try.

 

It also explains why when I put the earth-shadow boxes on my models, and then put the pivot to -10m, almost every shadow box cast a parallelogram across the next two tiles.

               
               

               
            

[Legacy_MerricksDad]

               

ok, that failed, so next I will try to confirm pivot to face double pass through. Let's see what we get. I bet there is a much higher degree of error there.

 

Edit:

 

testing the bottom of a tall mesh against the top of a tall mesh shows that the bottom and top need to be separated. Furthermore, any mesh element not adjacent to another element should probably be separated for good measure, or a test should be specifically performed on it to make sure it does not cancel shadows.

 

I may try another route. As SEN suggested, I can just take normal averages as a start. What I would like to do is get the face normal for all faces, separate them by similarity (already did that with the 6 directional shadows), then separate those by proximity, making kind of dirt clods in a pile, then make a pivot point at the average coordinates of the clod. Not so much pivot as actual mesh origin in relation to the verts. Changing the pivots on mine makes a terrible mess.

 

Ultimately, I am looking for something as accurate as full explosion, with the benefit of less mesh objects, and therefore less memory usage, less file space, and less export/import time. I think this clod method will do that. I may start with 500cm clod size and work down from there. Or, with the low concavity of these meshes I am working with, it might be super easy to get it in one try by using the transition height (200) as the clod size.

 

This feels like the reverse of mine craft, undoing voxels to find clods of shadow.

               
               

               
            

[Legacy_MerricksDad]

               

almost done '>

               
               

               
            

[Legacy_MerricksDad]

               

Here is how close I am:

 

 

So tomorrow, to finish up, all I need to do is a pivot point to face ray check to see if it passes through more than once face per ray. I think it will be sufficient to test only the pivot to face verts while the face being tested is removed. I'll also need to remove all faces sharing the vert being tested, as it is impossible with my mesh setup to hit one of those adjacent faces with the same ray. If the ray passes through the vert position and still strikes another face, then at least those faces removed for testing need to stay removed and be put into another mesh object.

 

Since I switched to using the walkmesh for shadow casting in the picture above, I can see very clearly exactly what is going on. From the pivot point/mesh origin at 0,0,0, which is between 600 and 1000cm below the actual mesh, and in the very middle of the x/y coordinates, the light gray line we are seeing is what you get when the shadow ray has touched two faces. The shape shown is the cancellation surface area, showing how much of both faces share a common ray volume. This shows me that at the area where the slope changes, there is at least one face which should be targeted from another direction.

 

Knowing this, I can probably glue all the 6 sub meshes back together and target specific problem faces instead of sorting them by direction first, though having 6 to start with doesn't seem to have any effect on shadow creation rates. In fact, when I was making shadows from 100% of the main mesh, the only thing that was slow was initial load time, and I've made character models which take longer to load than that! I guess what I am saying is that even the high poly nature of these tiles is still very low poly when it comes to shadow creation, which is very surprising to me given all the negative talk of shadows over the years. If I go with shadows based on the primary mesh, I'd certainly like some feedback on frame rates from users in the future. Or I can release an alternate set where the shadows are created from the walkmesh, as shown above. Alternates seem like a good thing. Options are always preferred over rigid dictation.

 

What I might end up doing is offsetting the North, South, East, and West directional meshes' origins by 250cm opposite the direction of the face normals. It seems like if I put the pivot point outside the 10m x/y primary region of the tile, it makes a shadow mess, which I assume is related similarly to the issue I found where anything less Z than the origin Z is truncated, so instead of finding a mathematically primed coordinate for the submesh origin, which would be off the tile region, I need to just fudge it within the 5m I can work with from the center. I've tried using exactly 5m, but that seems to make a mess, so I need to try smaller offsets, 4.9m or less from center.

 

I really hope to finish primary tile shadows tomorrow, but I have a few hours I need to be on the road for college stuff. It might not get done until tomorrow evening.

 

Next up is scripting a face collector for the walkmesh to detect which is walkable and which is not. Actually I already have that script, I just need to tweak the angles. Walkmesh will be followed directly by a face collector for the visible mesh which detects underlying walkmesh type, as well as what I like to call "rocks". I should be able to drop a ray from any face on the visible mesh and find the point on the walkmesh nearest it. Then I can detect which face I'm touching, and get its walkmesh type. This will tell me which kids of vegetation nodes I can place on such a face. I may end up doing a dummy node on every visible face with some details stored in the dummy variables. Maybe not.

 

About "rocks". If I can make a script which detects verts which are higher than surrounding faces, then I can determine what I can paint with boulder and outcrop textures, or otherwise cover that with decals via script. By selecting the prominent vertex and repeatedly expanding the face selection until a majority of newly selected faces level off or go back up, I can determine how large to paint such a rock. Kind of like flood-filling mesh knobs with rock texture. Time to get my rock outcrop decals ready!

               
               

               
            

[Legacy_MerricksDad]

               

Revelation! And Rage!

 

The negative shadows are not caused by an issue of pivot point, but the direct double pass through of faces by rays from the world light source. It has absolutely nothing to do with pivots being able to draw lines through a single face rather than two or more.

 

After I perfected the shadow models down to 100% accuracy based on pivots (or in my case object origin), some very few faces still have issues. If I draw a line from the negative shadow, up through nearby faces which should cause a shadow, toward the sun origin point, it passes through two faces on the same mesh.

 

To fix this, I need to calculate the exact angle of the sun in relation to the area, and once and for all determine if the sun is an object in a set coordinate system, or if the sun is an object with coordinate system per tile. I suspect it is per tile, because the shadows (and negatives) cast on each similar tile are, as far as I can see, exactly the same (and probably has some speed enhancing effects within the engine).

 

Once I can get the exact angle of the sun in areas, I can then use the sun as the checking pivot point, rather than the object origin, object pivot point, or whatever else I and many others have tried.

 

So ultimately, the answer was fairly simple, and possibly overlooked due to assumed mechanics. In reality, it was the angle of the sun the whole time. Now we can fix it, once and for all, and I can release a complex face checking script for exact shadow casting.

 

But... today I need to be out most of the day, so it won't likely get done soon.

 

Also, since the tile can be rotated in 4 directions, there needs to be 4 checkpoints passed before a shadow is totally validated. I have to make an assumption that the tile will be used in all 4 forms and then make sure no double pass through from the sun origin is detected. If it is, I just separate those faces to another object.

 

I fully suspect that the sun action is handled by a plane, rather than a point, which should make testing for the thing easy, because all I have to do is, from every face vert, test a ray up at an angle (same as the sun casts down) and see what faces it passes through. If the sun happens to be a point in worldspace, then the math will be impossible, because I cannot hope to calculate that for every tile at any distance from a 2x2 to 32x32 area. If instead the sun is a point per tilespace, it would still be difficult, but not as much so. I'm really really really hoping the sun is a plane, because then it won't matter if it is worldspace or tilespace, the plane will be the same, as will the angle.