About select_sampling_method
bbertolucci opened this issue · comments
Hi,
I am trying to understand IntegratedPifu code and trying to adapt it to my dataset who is not Thuman one.
I reach the select_sampling_method() in TrainDataset.py; and I think it's an important function to match the obj file with the image file.
So I have different questions :
# note, this is the solution for when dataset is "THuman"
# adjust sigma according to the mesh's size (measured using the y-coordinates)
y_length = np.abs(np.max(mesh.vertices, axis=0)[1]) + np.abs(np.min(mesh.vertices, axis=0)[1] )
sigma_multiplier = y_length/188
You wrote 188
but what does it represents exactly ? Is it an average y-length of all the Thuman models ? Is it the maximum y-length of all the Thuman models ? Or is it a random number ?
On the 3 free models of Thuman, I calculate the y length to 169, 171 and 187
-
What is the
compensation_factor = 4.0
? -
Is there a way to verify if my mesh are correctly oriented (in the same direction of the image) after applying the rotation (R) ?
-
Just a fix on mesh_util.py :
verts, faces, normals, values = measure.marching_cubes_lewiner(sdf, 0.5)
must be replaced by :
verts, faces, normals, values = measure.marching_cubes(sdf, 0.5, method="lewiner")
Thanks, and thank you also for your work it's so instructive !
Hi,
- You can use other number instead of 188, we picked 188 arbitrarily as it is one of the first numbers that worked for us.
compensation_factor
is to increase the number of sample points generated because, for instance, spatial sampling needs at leastself.num_sample_inout
sample points to be in each minibatch and the sample points in the minibatch has to be exactly 50% inside of the mesh and 50% outside of the mesh. There is thus a need to generate more sample points at the start and then filter the ones that we do not need later.- You can use the trimesh library (https://trimsh.org/trimesh.html) and download Meshlab to rotate or/and visualize the meshes.
- Yes, the new version of the package requires this, thank you for bringing this up.
Hi, thank you for you answer,
It's ok I totally understood your code !
But well there are still a lot of hardcoded values...
For example my dataset have obj files with a y max of "2" and not "188" (like Thuman), resulting the displacement of inner points is too strong and push the points outside ! :p.
In the same way depending on my dataset the depthmap value were too small too..
Well anyway I am currently regenerating my dataset to scale all of them to a max size of 188. It should work !
Thanks for the library, chatgpt gave me also some advise.
I am sharing it :
I added it to the end of : select_sampling_method()
mesh.visual = trimesh.visual.ColorVisuals(mesh, face_colors=[255, 0, 0, 25])
scene = trimesh.Scene(mesh)
def create_grid(size, spacing):
lines = []
for i in np.arange(-size/2, size/2 + spacing, spacing):
lines.append(trimesh.creation.cylinder(radius=0.01, segment=[(-size/2, 0, i), (size/2, 0, i)], cap_ends=False))
lines.append(trimesh.creation.cylinder(radius=0.01, segment=[(i, 0, -size/2), (i, 0, size/2)], cap_ends=False))
return lines
grid = create_grid(size=200, spacing=10)
for line in grid:
scene.add_geometry(line, geom_name="grid")
start_points_geom = trimesh.PointCloud(surface_points[:num_of_way_inside_pts], colors=[0, 255, 0, 255])
scene.add_geometry(start_points_geom, geom_name="test")
idx = 0
for point, direction in zip(surface_points, normal_vectors):
ray_points = point + direction * 20
ray_geom = trimesh.creation.cylinder(radius=0.1, segment=[point, ray_points], cap_ends=False, vertex_colors=[0, 0, 255, 255])
scene.add_geometry(ray_geom, geom_name="vector{}".format(idx))
idx += 1
if idx > num_of_way_inside_pts:
break
scene.show()
for i in range(num_of_way_inside_pts+1):
scene.delete_geometry("vector{}".format(i))
idx = 0
for point, direction in zip(surface_points, z_displacement):
ray_points = point + direction * 20
ray_geom = trimesh.creation.cylinder(radius=0.1, segment=[point, ray_points], cap_ends=False, vertex_colors=[0, 0, 255, 255])
scene.add_geometry(ray_geom, geom_name="displacement{}".format(idx))
idx += 1
if idx > num_of_way_inside_pts:
break
scene.show()
scene.delete_geometry("test")
for i in range(num_of_way_inside_pts+1):
scene.delete_geometry("displacement{}".format(i))
start_points_geom = trimesh.PointCloud( surface_points_with_normal_sigma, colors=[255, 255, 0, 255])
scene.add_geometry(start_points_geom, geom_name="test")
scene.show(point_size=1)
scene.delete_geometry("test")
inside_points_geom = trimesh.PointCloud(way_inside_pts, colors=[255, 0, 0, 255])
scene.add_geometry(inside_points_geom)
inside_points_geom = trimesh.PointCloud(surface_points[0:num_of_way_inside_pts], colors=[0, 255, 255, 255])
scene.add_geometry(inside_points_geom)
# Ajouter les points intérieurs (en green) à la scène
inside_points_geom = trimesh.PointCloud(outside_surface_points[0:num_of_way_inside_pts], colors=[0, 255, 0, 255])
scene.add_geometry(inside_points_geom)
idx = 0
for inside, outside in zip(way_inside_pts, outside_surface_points):
if not np.all(inside == 0) and not np.all(outside == 0):
ray_geom = trimesh.creation.cylinder(radius=0.1, segment=[inside, outside], cap_ends=False, vertex_colors=[0, 0, 255, 255])
scene.add_geometry(ray_geom, geom_name="displacement{}".format(idx))
idx += 1
if idx > num_of_way_inside_pts:
break
scene.show(point_size=1)