[HELP] Geometry instancing is not working

Question

[HELP] Geometry instancing is not working

StudenteChamp2 opened this issue 9 months ago · comments

I implemented geometry instancing inside my renderer. So multiple instances of a same mesh can share the same memory. I tried to do the same thing Embree side. Since now I have been creating an RTCGeometry(RTC_GEOMETRY_TYPE_TRIANGLE) for each mesh regardless they are instances or not.

I made a simple test. The scene has a single mesh that is a plane. That plane has an identity transform(its origin is (0, 0, 0) no rotation no scale). Then to render i perform intersect queries(rtcIntersect1) against the scene.

Without instancing I get the right result:

With instancing, intersection queries fails:

HERE THE CODE WITHOUT INSTANCING(WORKS):

// Initialize embree device.
g_device = rtcNewDevice(nullptr);

// Initialize the root scene.
g_scene = rtcNewScene(g_device);

// Read first mesh of test group --> the plane.
const auto group = Model::getMeshGroupPtr_FromEntity(targetEntity);
Model::Mesh* mesh = group->m_meshes[0];

// Create the root geometry.
g_geometry = rtcNewGeometry(g_device, RTC_GEOMETRY_TYPE_TRIANGLE);

// Set vertices. 
{
	/*struct FullVertex
	{
		Math::Vec3 position;
		Math::Vec3 normal;
		Math::Vec3 tangent;
		Math::Vec3 bitangent;
		Math::Vec2 texCoord;
	};*/
	const std::vector<FullVertex>& meshVertices = mesh->getVertices();

	rtcSetSharedGeometryBuffer(
		g_geometry,
		RTC_BUFFER_TYPE_VERTEX,
		0,
		RTC_FORMAT_FLOAT3,
		meshVertices.data(),
		0,
		sizeof(Graphics::FullVertex),
		meshVertices.size());
}

// Set indices.
{
	const std::vector<unsigned int>& meshIndices = mesh->getIndices();

	rtcSetSharedGeometryBuffer(
		g_geometry,
		RTC_BUFFER_TYPE_INDEX,
		0,
		RTC_FORMAT_UINT3,
		meshIndices.data(),
		0,
		3 * sizeof(unsigned int),
		meshIndices.size() / 3);
}

rtcCommitGeometry(g_geometry);
rtcAttachGeometryByID(g_scene, g_geometry, mesh->getFlatId());
rtcReleaseGeometry(g_geometry);

rtcCommitScene(g_scene);

// Init the intersection context.
m_context = new RTCIntersectContext();
rtcInitIntersectContext(m_context);`

HERE THE CODE WITH INSTANCING(DOESN'T WORKS). It is based on the instanced geometry tutorial.
Check the "NEW" tag to see what is different.

// Initialize embree device.
g_device = rtcNewDevice(nullptr);

// Initialize the root scene.
g_scene = rtcNewScene(g_device);

// Read first mesh of test group --> the plane.
const auto group = Model::getMeshGroupPtr_FromEntity(targetEntity);
Model::Mesh* mesh = group->m_meshes[0];

// Initialize the scene to use for instancing.
g_scene1 = rtcNewScene(g_device);// NEW

// Create the root geometry.
g_geometry = rtcNewGeometry(g_device, RTC_GEOMETRY_TYPE_TRIANGLE);

// Set vertices. 
{
	/*struct FullVertex
	{
		Math::Vec3 position;
		Math::Vec3 normal;
		Math::Vec3 tangent;
		Math::Vec3 bitangent;
		Math::Vec2 texCoord;
	};*/
	const std::vector<FullVertex>& meshVertices = mesh->getVertices();

	rtcSetSharedGeometryBuffer(
		g_geometry,
		RTC_BUFFER_TYPE_VERTEX,
		0,
		RTC_FORMAT_FLOAT3,
		meshVertices.data(),
		0,
		sizeof(Graphics::FullVertex),
		meshVertices.size());
}

// Set indices.
{
	const std::vector<unsigned int>& meshIndices = mesh->getIndices();

	rtcSetSharedGeometryBuffer(
		g_geometry,
		RTC_BUFFER_TYPE_INDEX,
		0,
		RTC_FORMAT_UINT3,
		meshIndices.data(),
		0,
		3 * sizeof(unsigned int),
		meshIndices.size() / 3);
}

rtcCommitGeometry(g_geometry);
rtcAttachGeometryByID(g_scene1, g_geometry, mesh->getFlatId());// NEW
rtcReleaseGeometry(g_geometry);

rtcCommitScene(g_scene1); // NEW

// Create an instance. All block is NEW.
{
	g_instance0 = rtcNewGeometry(g_device, RTC_GEOMETRY_TYPE_INSTANCE);
	rtcSetGeometryInstancedScene(g_instance0, g_scene1);

	//const glm::mat4& transform = mesh->getTransform()->getMatrix();
	//rtcSetGeometryTransform(g_instance0, 0, RTC_FORMAT_FLOAT4X4_COLUMN_MAJOR, (float*)&transform[0][0]);
	const float transform[] = { 1.0f, 0.0f, 0.0f, 0.0f,
						 0.0f, 1.0f, 0.0f, 0.0f,
						 0.0f, 0.0f, 1.0f, 0.0f,
						 0.0f, 0.0f, 0.0f, 1.0f };

	rtcSetGeometryTransform(g_instance0, 0, RTC_FORMAT_FLOAT4X4_COLUMN_MAJOR, &transform[0]);

	rtcAttachGeometry(g_scene, g_instance0);
	rtcReleaseGeometry(g_instance0);
}

rtcCommitScene(g_scene);

// Init the intersection context.
m_context = new RTCIntersectContext();
rtcInitIntersectContext(m_context);

The issue is present on both Embree v3.1.0 and v4.3.0. So it is Embree bug or am I doing something wrong?

Sven Woop · Answer 1 · Tue Oct 17 2023 13:38:32 GMT+0800 (China Standard Time)

What geometry is the mesh supposed to be? It is likely just the one grass object, which is also seen in the instancing case. I cannot spot any obvious error in your code.

Florian Reibold · Answer 2 · Tue Oct 17 2023 21:48:27 GMT+0800 (China Standard Time)

You can also take a look at the instance array geometry tutorial

https://github.com/embree/embree/blob/master/tutorials/forest/forest_device.cpp

that we added in Embree 4.3.0. This tutorial does almost the same as you want to do but with trees instead of grass/herbs.

StudenteChamp2 · Answer 3 · Wed Oct 18 2023 06:08:21 GMT+0800 (China Standard Time)

@svenwoop @freibold Thanks for helping.
I updated the question because it could be confusing. So now the scene has only one mesh: the plane.

To recall I have 2 implementations. I don't get why the second is not working.

1) No instancing(WORKS)

Steps:
-->Create the scene(rtcNewScene)
-->Create a RTCGeometry(RTC_GEOMETRY_TYPE_TRIANGLE) for the plane
-->Set the geometry vertex and index buffers(using rtcSetSharedGeometryBuffer).
-->Commit the geometry(rtcCommitGeometry)
-->Attach the geometry to the scene(rtcAttachGeometryByID)
-->Commit the scene.

2) Instancing(DOESN'T WORK)

Steps based on the instanced geometry tutorial :
-->Create the root scene
g_scene = rtcNewScene(g_device)

-->Create the instanced scene
g_scene1 = rtcNewScene(g_device)

-->Create a RTCGeometry for the plane
g_geometry = rtcNewGeometry(g_device, RTC_GEOMETRY_TYPE_TRIANGLE);

-->Set the geometry vertex and index buffers.

-->Commit the geometry
rtcCommitGeometry(g_geometry)

-->Attach the geometry to the instanced scene
rtcAttachGeometryByID(g_scene1, g_geometry, mesh->getFlatId())

--> Commit the instanced scene
rtcCommitScene(g_scene1)

-->Create a geometry instance.
g_instance0 = rtcNewGeometry(g_device, RTC_GEOMETRY_TYPE_INSTANCE)

-->Link the geometry instance to the instanced scene
rtcSetGeometryInstancedScene(g_instance0, g_scene1)

-->Set the instance transform
`
const float transform[] = { 1.0f, 0.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f, 0.0f,
0.0f, 0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 0.0f, 1.0f };

rtcSetGeometryTransform(g_instance0, 0, RTC_FORMAT_FLOAT4X4_COLUMN_MAJOR, &transform[0]))`

-->Attach the instance to the root scene
rtcAttachGeometry(g_scene, g_instance0)

-->Commit the root scene
rtcCommitScene(g_scene))

Sven Woop · Answer 4 · Wed Oct 18 2023 12:32:18 GMT+0800 (China Standard Time)

Could you please provide a reproducer by modifying the instance geometry tutorial? It is not obvious what is wrong there and we would need a small reproducer to debug this.

StudenteChamp2 · Answer 5 · Sun Oct 22 2023 08:05:29 GMT+0800 (China Standard Time)

I managed to have it working by doing the same steps as what it is in the instanced geometry tutorial. So it works but I don't understand why. It look like this:

-->Create the root scene
g_scene = rtcNewScene(g_device)

-->Create the instanced scene
g_scene1 = rtcNewScene(g_device)

-->Create a RTCGeometry for the plane
g_geometry = rtcNewGeometry(g_device, RTC_GEOMETRY_TYPE_TRIANGLE);

-->Set the geometry vertex and index buffers.

-->Commit the geometry
rtcCommitGeometry(g_geometry)

-->Attach the geometry to the instanced scene
rtcAttachGeometryByID(g_scene1, g_geometry, mesh->getFlatId())

-->Release the geometry
rtcReleaseGeometry(g_geometry);

--> Commit the instanced scene
rtcCommitScene(g_scene1)

-->Create a geometry instance.
g_instance0 = rtcNewGeometry(g_device, RTC_GEOMETRY_TYPE_INSTANCE)

-->Attach the geometry instance to the root scene
rtcAttachGeometry(g_scene, g_instance0)

-->Release the geometry instance
rtcReleaseGeometry(g_instance0);

-->Set the instance transform
`
const float transform[] = { 1.0f, 0.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f, 0.0f,
0.0f, 0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 0.0f, 1.0f };

rtcSetGeometryTransform(g_instance0, 0, RTC_FORMAT_FLOAT4X4_COLUMN_MAJOR, &transform[0]))`

--> Commit the instance geometry
rtcCommitScene(g_instance0)

-->Commit the root scene
rtcCommitScene(g_scene))

StudenteChamp2 · Answer 6 · Sun Oct 22 2023 08:13:42 GMT+0800 (China Standard Time)

@svenwoop How does rtcReleaseGeometry works?
Here we create the instance geometry then we free it inside the same function(device_init ):
https://github.com/embree/embree/blob/master/tutorials/instanced_geometry/instanced_geometry_device.cpp#L123

Then we later use the same geometry(we released) and we commit it:
https://github.com/embree/embree/blob/master/tutorials/instanced_geometry/instanced_geometry_device.cpp#L339

Sven Woop · Answer 7 · Mon Oct 23 2023 13:10:39 GMT+0800 (China Standard Time)

The rtcReleaseGeometry decreases the reference counter on the geometry handle, thus you should not use that handle anymore. However, if you previously attach the handle to some scene, then the object itself will still be alive, as the scene itself increases the reference count on that geometry.