SimSwap without replacing the mouth

[SamPse]

Is there a way to use SimSwap without replacing the mouth? only Eyes and nose !

[Thomas De]

You can play with the face parts in util/reverse2original.py
Remove #1 which is the face
You can find the list of facial parts somewhere in the original github repo

[SamPse]

You can play with the face parts in util/reverse2original.py Remove #1 which is the face You can find the list of facial parts somewhere in the original github repo

Thank you for your answer. I am no expert with opencv. The part to modify is this if I understood correctly.

def encode_segmentation_rgb(segmentation, no_neck=True):
parse = segmentation

face_part_ids = [1, 2, 3, 4, 5, 6, 10, 12, 13] if no_neck else [1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14]
mouth_id = 11
# hair_id = 17
face_map = np.zeros([parse.shape[0], parse.shape[1]])
mouth_map = np.zeros([parse.shape[0], parse.shape[1]])
# hair_map = np.zeros([parse.shape[0], parse.shape[1]])

for valid_id in face_part_ids:
    valid_index = np.where(parse==valid_id)
    face_map[valid_index] = 255
valid_index = np.where(parse==mouth_id)
mouth_map[valid_index] = 255
# valid_index = np.where(parse==hair_id)
# hair_map[valid_index] = 255
#return np.stack([face_map, mouth_map,hair_map], axis=2)
return np.stack([face_map, mouth_map], axis=2)

[Thomas De]

Yes. Just remove 1 and ,
you have to use the --use_mask option in your commandline.

[SamPse]

Yes. Just remove 1 and , you have to use the --use_mask option in your commandline.

Thanks !
I removed 1 and , forme this : face_part_ids = [2, 3, 4, 5, 6, 10, 12, 13] if no_neck else [2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14]
but doesn’t work.

[Thomas De]

Yes, mask has soft border....
You can cange iterations from 7 to 1 at line 81
smooth_mask = SoftErosion(kernel_size=17, threshold=0.9, iterations=7).cuda()

Look at the image, from left to right:
Mask with #1 , without #1 and iterations 7 , without #1 but iterations 1

EDIT: forgot to say, you can play also with kernel size, only odd numbers I think

[SamPse]

Yes, mask has soft border.... You can cange iterations from 7 to 1 at line 81 smooth_mask = SoftErosion(kernel_size=17, threshold=0.9, iterations=7).cuda()

Look at the image, from left to right: Mask with #1 , without #1 and iterations 7 , without #1 but iterations 1

EDIT: forgot to say, you can play also with kernel size, only odd numbers I think

Very interesting but on my side I have no change. I don't know why, just for information I use the graphical version: test_video_swapsingle_GUI1.py

[Thomas De]

Ok. The GUI version is not official. I guess you donwloaded it from my youtube channel?
I will check it the next few days

[SamPse]

Ok. The GUI version is not official. I guess you donwloaded it from my youtube channel? I will check it the next few days

Yes it's from your youtube channel. Thank you!

[Thomas De]

I've tried it myself. It doesn't work as expected.
Maybe it's because they they do masking complety different as I do it in my other projects.
If I remember correctly, in the initial version of simswap (2021) it did work this way?

Sorry, maybe it requires more code changes...