Giters

strawberryfg / int-3dhuman-I1

This is a personal fun implementation of I1 (heatmap + integral loss) in Integral Regression. Trained only on H36M w/ 2-stage coarse to fine Hourglass. JUST FOR REFERENCE.

Repository from Github https://github.comstrawberryfg/int-3dhuman-I1Repository from Github https://github.comstrawberryfg/int-3dhuman-I1

int-3dhuman-I1

Hey there, this is the implementation of I1 (heatmap loss + integral loss) in this paper. Training is on H36M exclusively with 2-stage coarse to fine Hourglass. Kindly jump to Training procedure below if you are not using C++.

Sketch

${POSE_ROOT}
+-- caffe_code
+-- data
+-- figs
+-- models
+-- training
+-- testing
+-- README.md

External links (closely related projects)

H1

I1 with group normalization, batch size = 1

Source code & Installation

Please refer to the "caffe_code/" folder and "Installation" section in readme of this repo

Configuration

This is for Caffe Ubuntu with 12 GB Titan card.

Data

Same as "data/". Note this is different from commonly used data like effective baseline. Major difference is my training(testing) set has 1559571(548999) images.

MPJPE Performance

d2 MPJPE(mm) of this repo (I1) Caffe Model (For reference) MPJPE(mm) of corresponding H1 repo
16 71.0 net_iter_526322.caffemodel 73.6
32 55.9 net_iter_567946.caffemodel 68.6
64 65.9 net_iter_671782.caffemodel 67.1

Note that in order to restrain one from tuning loss weight of 2D/3D HM/integral to the best of one's ability, adaptive euclidean loss weight balancing technique (as detailed below) is initialized insofar as it does not degrade performance.

I have a deep skepticsm concerning the bearing of different training technique on final number. And so the point I would like to make here is that what matters most is the algorithm itself rather than the superiority or inferiority of number.

As to depth dimension d2, the original paper uses 64. This paper uses 16. This repo prefers 32.

Piece of advice here, insofar as d2 is at least 16, it's okay.

Training procedure

First off, THERE IS NO GUARANTEE THAT OTHER TRAINING STRATEGY WOULD NOT YIELD SAME/SIMILAR OR BETTER RESULTS. THIS IS JUST FOR REFERENCE.

Having said that, I WOULD BE MORE THAN HAPPY TO KNOW IF ANYONE HAS A MORE ELEGANT WAY TO TRAIN THIS WACKY STUFF. MUCH APPRECIATED. THANKS!

Back to our topic,

The starting point is MPII pretrained model. See head of training section .

Note 1 [heatmap2 init std]: The init of layer "heatmap2" (which reduces dimension to generate final 3d heatmap output) is gaussian with standard deviation of a hyper param.

Note 2 [heatmap2_flat_scale]: Before softmax normalization, hereinafter a scale 30.0 is multiplied by 3d heatmap output for numerical reason. The semantic of heatmap no longer preserves any more, as alluded in this issue. Otherwise I have no idea how to train I1. CAN ANYONE ENLIGHTEN ME?

Note 3 [loss]: Adaptive weight balancing is employed. Details here. Found this paper several weeks after writing the AdaptiveWeightEucLossLayer

  1. Adaptive H1 adaptively computes gradient magnitude of 2D/3D heatmap w.r.t neuron, and tries to balance gradients flowing from 2D and 3D heatmap. Search AdaptiveWeightEucLoss in this pdf for expatiation.

  2. Adaptive I1 adaptively balances the gradient of 2D/3D heatmap/integral loss w.r.t neuron, whereupon restraining one from tuning weight ratio between heatmap euclidean loss and integral regression loss.

  3. Manual sets loss ratio of heatmap:integral manually after Adaptive I1 warm-up.

The following is sorted by d2 (depth dimension of 3d heatmap) in increasing order.

Testing steps

Put corresponding trained models under folder models. (Simply scroll up to MPJPE Performance for names of models)

d2 = 32

cd testing/d2=32
$CAFFE_ROOT/build/tools/caffe test -model test_d32_statsfalse.prototxt -weights models/net_iter_567946.caffemodel	-iterations 274500

Denouement: 55.9 mm

d2 = 64

cd testing/d2=64
$CAFFE_ROOT/build/tools/caffe test -model test_d64_statsfalse.prototxt -weights models/net_iter_671782.caffemodel	-iterations 274500

Denouement: 65.9 mm

d2 = 16

cd testing/d2=16
$CAFFE_ROOT/build/tools/caffe test -model test_d16_statsfalse.prototxt -weights models/net_iter_526322.caffemodel	-iterations 274500

Denouement: 71.0 mm

FAQ

Glad to answer any concern. Just email strawberryfgalois@gmail.com. Thanks!

About

This is a personal fun implementation of I1 (heatmap + integral loss) in Integral Regression. Trained only on H36M w/ 2-stage coarse to fine Hourglass. JUST FOR REFERENCE.