When should we mark the 'shouldRemove' sign on a contour vertex?

[Michael Tang]

static unsigned char getCornerHeight(dtTileCacheLayer& layer,
                                     const int x, const int y, const int z,
                                     const int walkableClimb,
                                     bool& shouldRemove)
{
	const int w = (int)layer.header->width;
	const int h = (int)layer.header->height;

	int n = 0;

	unsigned char portal = 0xf;
	unsigned char height = 0;
	unsigned char preg = 0xff;
	bool allSameReg = true;

	for (int dz = -1; dz <= 0; ++dz)
	{
		for (int dx = -1; dx <= 0; ++dx)
		{
			const int px = x+dx;
			const int pz = z+dz;
			if (px >= 0 && pz >= 0 && px < w && pz < h)
			{
				const int idx  = px + pz*w;
				const int lh = (int)layer.heights[idx];
				if (dtAbs(lh-y) <= walkableClimb && layer.areas[idx] != DT_TILECACHE_NULL_AREA)
				{
					height = dtMax(height, (unsigned char)lh);
					portal &= (layer.cons[idx] >> 4);
					if (preg != 0xff && preg != layer.regs[idx])
						allSameReg = false;
					preg = layer.regs[idx];
					n++;
				}
			}
		}
	}

	int portalCount = 0;
	for (int dir = 0; dir < 4; ++dir)
		if (portal & (1<<dir))
			portalCount++;
	shouldRemove = false;
	if (n > 1 && portalCount == 1 && allSameReg)
	{
		shouldRemove = true;
	}

	return height;
}

I am studing the code in the phase of building contour of a TileLayer. In the "getCornerHeight()" Function, as you can see the code is above, I cant not figure out when we should mark the "shouldRemove" sign true? How does it happen when the number of the walkable spans around the vertex are more than 2 at the same time they have the same direction to another layer?

i have tested some demo scenes and cannot see it happened.

thanks for helping...

[Mikko Mononen]

I tries to capture the case where there is an extra vertex along a portal edge. Portal edge is an edge that leads to neighbour tile. I can happen e.g. in this kind of case:

+---x-x---+
|   : :   |
|   +-+   |
|   | |   |
|   +-+   |
|   : :   |
+---x-x---+

The vertices marked by x should be marked to be removed. Assuming all the areas are same type. Those extra vertices are there due to the area partitioning.

[Michael Tang]

Thank you very much. I did not expect this to happen.