weird geometry & texture on leg

[ZunDuJiaDu]

Anyone met the same weird geometry and texture on the leg?

I guess this is caused by the mismatch of SMPLX model and provided images. Do you have suggestions?

[Yangyi Huang]

Yes, this can happen when SMPL-X estimates are inaccurate. You can try passing a extra loss mask to mask out all the normal regularization loss and reconstruction losses in the incorrect area, in this part of the code

TeCH/core/lib/trainer.py

Lines 242 to 246 in 5efe7d6

 if self.cfg.data.loss_mask is not None: 
     self.loss_mask = np.array(Image.open(self.cfg.data.loss_mask).resize(self.input_image.shape[1:]))[..., -1] / 255 
     self.loss_mask_norm = np.array(Image.open(self.cfg.data.loss_mask).resize((512, 512)))[..., -1] / 255 
     self.loss_mask = torch.tensor(self.loss_mask, dtype=torch.float).to(self.device).unsqueeze(0) * self.input_mask 
     self.loss_mask_norm = torch.tensor(self.loss_mask_norm, dtype=torch.float).to(self.device).unsqueeze(0) * self.normal_mask 

[Subin_Kim]

[Origin image]

[Mesh]

Thank you so much.
I want an outstretched leg on mesh. But, I got a folded leg like image.
Any suggestion for this error?

[Yangyi Huang]

The SMPL-X pose estimation method (PIXIE) we use often encounter such problem, you can try to replace the smplx estimation with Pymaf-x or other more advanced methods