About generating landmark heatmap offset groud truth label
RobbertOnly opened this issue · comments
Hello Emiliana emilianavt:
Very nice work for this project! And i have some problems about landmark heatmap offset label for training:
1. How to generate groud truth?
2. What is the loss function for landmark heatmap offset?
I am looking forward to your reply.Thank you very much!
Hi, thank you!
- For generating the offset ground truth, you basically perform the inverse calculations given here. The result will look like a horizontal and vertical gradient from 0 to 1. If the ground truth landmark happens to be on exactly one of the pixel columns/rows after downscaling, it should have the value 0.5.
- Adaptive wing loss as given in model.py is used for this as well as the main heatmap.
what is the rationale for the x, y offset gradient heatmap? I did not see Adaptive wing loss using this. Is is a thing that you invented?
The offset maps allow the UNet part to upscale to a lower resolution than the input resolution while still giving high precision outputs. Without it, landmarks would have only 1/4 resolution per dimension.
It was described in both the ULSee document and the Real-Time Human Pose Estimation one linked here.