Runtime Error

Question

Runtime Error

hanssssssss opened this issue 3 years ago · comments

Hi! Thanks for sharing your great work. I installed your pt_mesh_render and tested it successfully. But when I used it to render the mesh ,something went wrong.
The errors are below, Do you konw what cauesd these errors ?

Traceback (most recent call last):
File "E:/PycharmProjects/pytorch_face/Render_test.py", line 77, in
im_rgba = Render_block(face_shape,face_norm,face_color,camera_scale,f_scale,facemodel,1)
File "E:/PycharmProjects/pytorch_face/Render_test.py", line 59, in Render_block
far_clip=50.0)
File "E:\PycharmProjects\pytorch_face\pt_mesh_renderer\mesh_renderer.py", line 355, in mesh_renderer
image_width, image_height, [-1] * vertex_attributes.shape[2])
File "E:\PycharmProjects\pytorch_face\pt_mesh_renderer\rasterizer.py", line 112, in rasterize_clip_space
image_width, image_height)
File "E:\PycharmProjects\pytorch_face\pt_mesh_renderer\RasterizeTriangles.py", line 66, in forward
barycentric, triangle_ids, z_buffer = forward_function(vertices, triangles, image_width, image_height)
RuntimeError

hanssssssss · Answer 1 · Tue Feb 23 2021 13:37:28 GMT+0800 (China Standard Time)

These are my codes, I wanted to use it to render a human face.
`
def Render_block(face_shape , face_norm , face_color , camera_scale , f_scale ,facemodel,batchsize =1, is_train = None ):

# render reconstruction images
n_vex = int(facemodel.idBase.shape[0]/3)
fov_y = 2*torch.atan(torch.tensor(112./(1015*f_scale)))*180./m.pi
fov_y = torch.reshape(fov_y , [batchsize])

# full face region [batch_size, vertex_count, 3]
face_shape = torch.reshape(face_shape,[batchsize,n_vex,3])
face_norm = torch.reshape(face_norm,[batchsize,n_vex,3])
face_color = torch.reshape(face_color,[batchsize,n_vex,3])

# setting camera settings
camera_position = torch.tensor([[0,0,10.0]],dtype=torch.float32)*torch.reshape(camera_scale,[-1,1])
camera_lookat = torch.tensor([0,0,0.0],dtype=torch.float32)
camera_up = torch.tensor([0,1.0,0],dtype=torch.float32)

# setting light source position
#(intensities are set to 0 because we have computed the vertex color)
light_position = torch.tensor([0,0,1e5],dtype=torch.float32)
light_position = (torch.reshape(light_position,[1,1,3])).repeat([batchsize,1,1])
light_intensities = torch.tensor([0.0,0.0,0.0],dtype=torch.float32)
light_intensities = (torch.reshape(light_intensities, [1, 1, 3])).repeat([batchsize, 1, 1])
ambient_color = torch.tensor([1.0, 1, 1], dtype=torch.float32)
ambient_color = (torch.reshape(ambient_color, [1,3])).repeat([batchsize,1])

face_shape  =face_shape.cuda()
face_norm = face_norm.cuda()
face_color = face_color.cuda()
camera_position = camera_position.cuda()
camera_lookat = camera_lookat.cuda()
camera_up = camera_up.cuda()
light_position = light_position.cuda()
light_intensities = light_intensities.cuda()
ambient_color = ambient_color.cuda()
fov_y = fov_y.cuda()

img_rgba = mesh_renderer.mesh_renderer(vertices = face_shape,
                                       triangles = (facemodel.face_buf-1).int().cuda(),
                                       normals = face_norm,
                                       diffuse_colors = face_color,
                                       camera_position = camera_position,
                                       camera_lookat = camera_lookat,
                                       camera_up= camera_up,
                                       light_positions = light_position,
                                       light_intensities = light_intensities,
                                       image_width = 224,
                                       image_height =224,
                                       fov_y = fov_y,
                                       ambient_color= ambient_color,
                                       far_clip=50.0)


img_rgba = torch.clamp(img_rgba, 0, 255).float()

if name == 'main':

face = loadmat('./output/000002.mat')

face_shape = torch.from_numpy(face['faceshape'])

face_color = torch.from_numpy(face['face_color_'])
face_texture = torch.from_numpy(face['face_texture_'])
face_norm = torch.from_numpy(face['face_norm_'])
camera_scale = torch.from_numpy(face['camera_scale'])
f_scale = torch.from_numpy(face['f_scale'])

facemodel = BFM()
im_rgba = Render_block(face_shape,face_norm,face_color,camera_scale,f_scale,facemodel,1)
plt.imshow(im_rgba)
plt.show()`

Tianyang Shi · Answer 2 · Mon Mar 01 2021 14:57:53 GMT+0800 (China Standard Time)

Hi, I'm sorry for this delay response.

I hope you have solved your problem. If not, you may check the value and size of vertices and triangles. In my experience, the index in triangles should start from 0 rather than 1, where the later one may cause your issue.

hanssssssss · Answer 3 · Mon Mar 01 2021 15:22:33 GMT+0800 (China Standard Time)

Hi! Thanks for your reply. I did use triangles = (facemodel.face_buf-1).int().cuda(), to make suret the indexs start from 0. But the issue is still there. Do you have any other idea about this?
Thanks a lot.

Tianyang Shi · Answer 4 · Mon Mar 01 2021 15:32:10 GMT+0800 (China Standard Time)

Hi, perhaps you may convert triangles to a LongTensor? I think this part is type-sensitive.

hanssssssss · Answer 5 · Mon Mar 01 2021 15:55:17 GMT+0800 (China Standard Time)

I have tried your method, but the error still there. I have no idea why this Runtime error show up.😥

Tianyang Shi · Answer 6 · Mon Mar 01 2021 16:03:13 GMT+0800 (China Standard Time)

Could you please upload a demo? Maybe I can debug it on my PC.

hanssssssss · Answer 7 · Mon Mar 01 2021 17:25:36 GMT+0800 (China Standard Time)

Thanks!!!! I really appreciate your help.

This is the demo and related information.
render_test.zip

Tianyang Shi · Answer 8 · Mon Mar 01 2021 18:29:02 GMT+0800 (China Standard Time)

Hi, I may solve your problem, which may be caused by a weird memory discontinuity. You can add .contiguous() on your face_buf. For example: face_buf = torch.from_numpy(face_buf_reader['face_buf']).contiguous()

hanssssssss · Answer 9 · Mon Mar 01 2021 18:41:06 GMT+0800 (China Standard Time)

Thanks for your patience and your help!!!!! I will try it.

hanssssssss · Answer 10 · Mon Mar 01 2021 22:53:35 GMT+0800 (China Standard Time)

I followed your instruction and finally solved this problem. Thanks!
I have another question.When I get the output of the mesh_render(a rgba image). I used clamp(0-255) to and slice to get a RGB image,but what I get is a image with color of black and white. Is this a normal situation?

Tianyang Shi · Answer 11 · Mon Mar 01 2021 22:57:45 GMT+0800 (China Standard Time)

This is a common issue for plt.imshow. You may use 0-1 in float or 0-255 in uint8, while clamp function does not change the type of image. For your case, you may use plt.imshow(img.astype("uint8")) to solve this problem.

hanssssssss · Answer 12 · Mon Mar 01 2021 23:01:58 GMT+0800 (China Standard Time)

Okay!!! Thanks for your help and your great work!!!!!!!!