DirtyHarryLYL / Transferable-Interactiveness-Network

Hello, please bother you. I have a small question for you. About self.spatial = tf.placeholder(tf.float32, shape=[None, 64, 64, 3], name ='sp') in your code. You changed the 2 in the code of this paper of ICAN to 3. I would like to ask what is the reason for this change? Thank you.

The first two channels of self.spatial in TIN are the same as that in iCAN. The third channel added is pose map, as shown in Fig. 4 in the paper.

Hello, I tried to follow your method and also changed the 2 channel in ICAN to 3 channel, but the following error occurred in the test.I would like to ask you, did I make a mistake during training or did I need to change the code during testing?thank you！

Pre-trained weights loaded.
2020-07-09 23:16:43.136085: W tensorflow/core/framework/op_kernel.cc:1152] Invalid argument: You must feed a value for placeholder tensor 'sp' with dtype float
[[Node: sp = Placeholderdtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]

Did you make sure that in the feed_dict, blobs['sp'] has size of [n, 64, 64, 3] when you change the two channel in iCAN to 3 channel?

Thank you for your answer.I've changed that, sp is the third pathway, and I've added the first to the body posture estimate.So I made a mistake here, right?

About feed_dict, I modified it in test_VCOCO. Is that right?Blobs [' sp] = Get_next_sp_with_pose (Human_out [2], the Object [2], Human_out [6]), reshape (1, 64, 64, 3)

If you are using checkpoint from iCAN and trying to change the channel of sp from 2 to 3, retraining of iCAN is required, since the original iCAN weight doesn't match the changed channel. If I misunderstood please point out.
Or you could check

Transferable-Interactiveness-Network/lib/networks/TIN_HICO.py

Line 186 in b29d702

    
           conv1_sp      = slim.conv2d(self.spatial[:,:,:,0:2], 64, [5, 5], padding='VALID', scope='conv1_sp')

, where iCAN weight could be used without retraining since only the first two channels are used for the pathway involving iCAN weight.

Thank you for your answer. Well, I'm trying to fuse the characteristics of the gesture estimation with the human pathway and the interaction pathway.And I've retrained.But there was an error in the test.So I think I made a mistake increasing the number of channels, but I didn't know what to do with it.

Noticing your error message, probably the mistake lies in the data type? Since it mentions data type instead of data shape.

You may be correct, but I made the modification with reference to your TIN paper, and I am not clear where to modify this data type.

self.spatial = tf.placeholder(tf.float32, shape=[None, 64, 64, 3], name = 'sp')