How can we get the X, Y, Z coordinates for 3d pose predictions?
ShaminiKoravuna opened this issue · comments
Hello everyone,
I'm trying to implement this code by adding the foot keypoints in the dataset so that it can be further used for gait analysis. Can I modify it with the perprocessed dataset? If so what are the exact key joint numbers I should consider. I have searched it everywhere but I couldn't find the exact keyjoints configuration anywhere.
Thanks in advance
The order of joints in this case does not matter. You can place it in any order as long as you remember which position corresponds to which joint for visualization. You can modify the preprocessed dataset and make your 2d joint predictor predict more than 16 points given you have the labels for other keypoints.
Thanks a lot for the quick response I'll do that.
How can I get the X, Y, Z coordinates for 3d pose predictions?
create_movie.py line 204
for i in range(n2d):
#t0 = time()
print("Working on figure {0:04d} / {1:05d}... \n".format(i+1, n2d), end='')
p2d = enc_in[i,:]
im = Image.open( fnames[i] )
ob1.update(im,p2d)
# Plot 3d gt
p3d = poses3d[i,:]
ob2.update(p3d)
fig.canvas.draw()
img_str = np.fromstring (fig.canvas.tostring_rgb(), np.uint8)
ncols, nrows = fig.canvas.get_width_height()
nparr = np.fromstring(img_str, dtype=np.uint8).reshape(nrows, ncols, 3)
#img_np = cv2.imdecode(nparr, cv2.CV_LOAD_IMAGE_COLOR)
print(FLAGS.output_dir+'{0:05d}.jpg'.format(i+1))
cv2.imwrite(FLAGS.output_dir+'{0:05d}.jpg'.format(i+1), nparr[:,:,::-1])
print(p3d)
when I tried print (p3d) it was showing a vector like this:
and the vector is the same for all the frames.
What is the dimension of the 3D pose vector?
it's 96
sorry for the trouble I got it now. As there 96 values in the vector these values correspond to x, y, z values of the 32 key joints i.e., 32*3=96 I tried plotting it in 3D matplotlib if was showing all the key joints. Thanks a lot for you help.
I was trying the get the foot tip key joints. I made some changes in viz.py for plotting the Foot tip but the two-foot joints are getting merged.
viz.py:
class Ax3DPose(object):
def __init__(self, ax, lcolor="#3498db", rcolor="#e74c3c"):
## REMOVE 15 from I and J
self.I = np.array([1,2,3,4,1,7,8,9,1, 13,14,15,14,18,19,14,26,27])-1
self.J = np.array([2,3,4,5,7,8,9,10,13,14,15,16,18,19,20,26,27,28])-1
self.LR = np.array([1,1,1,1,0,0,0,0,0, 0, 0, 0, 0, 0, 0, 1, 1, 1], dtype=bool)
self.ax = ax
Can anyone help me out with this?
Thanks in Advance
This seems like an error in connecting the keypoints. You are connecting to the same foot for both legs. Please check whether you are creating the connection matrix properly.
sorry for the trouble I got it now. As there 96 values in the vector these values correspond to x, y, z values of the 32 key joints i.e., 32*3=96 I tried plotting it in 3D matplotlib if was showing all the key joints. Thanks a lot for your help.
I have plotted them according to this format [x1,y1,z1,x2,y2,z2,...] and I got the output like is this the right format?
because for one leg it is showing all the joints and for the other, some joints are missing.
This seems like an error in connecting the keypoints. You are connecting to the same foot for both legs. Please check whether you are creating the connection matrix properly.
And for getting the foot I was considering joints (4,5) and (9,10). 4 and 9 are the toe joints and 5 and 10 are specified as site joints in Human 3.6m data. If you have any idea can you please tell me what those site joints are and how they are determining those site joints?
@ShaminiKoravuna could you explain about 32 keypoints? I am trying my best to grasp how are these 16 detected keypoints from Stacked Hourglass are saved as 64 and then read 32 only?
HI @ShaminiKoravuna. Sorry for late response.
I have plotted them according to this format [x1,y1,z1,x2,y2,z2,...] and I got the output like is this the right format?
It looks like you are using the right format. That's how the coordinates are arranged.
because for one leg it is showing all the joints and for the other, some joints are missing.
Hmm. This is strange. It may be a misdetection or an error in estimating the 3D points. Have you tried some other image / 2D pose? You can color the left and right joints differently so that it is easy to understand which joints are correctly estimated.
And for getting the foot I was considering joints (4,5) and (9,10). 4 and 9 are the toe joints and 5 and 10 are specified as site joints in Human 3.6m data. If you have any idea can you please tell me what those site joints are and how they are determining those site joints?
As far as I can remember, site joints were used for calculating the x,y, and z coordinates of other joints from the kinematic tree. By default, the data in Human3.6M is in axis-angle format. Site joints kind of act as the leaf of the kinematic tree.
@noumanriazkhan I think she trained the stacked hourglass network from scratch to predict 32 joints.
Hi @rayat137 but the h5 file you provided, when read in emc_in and selected dims_to_use, its also 32 keypoints while Hourglass network provides 16...?
Hi @rayat137 but the h5 file you provided, when read in emc_in and selected dims_to_use, its also 32 keypoints while Hourglass network provides 16...?
Even I have the same question did you figure it out?
Yes, so its flatten. You can reshape it with reshape(-1,3)
Note: It's been a while I worked on this but I believe this is what I did.
Thanks a lot for the quick response @noumanriazkhan .. Is it taken for 32 joints or for 16?
I'm getting this error when I'm trying to estimate it with the predictions from my own images
linear_size = 1024
Loading model
Model created
[[[ 31. 58.5]
[ 29. 43.5]
[ 31. 32.5]
[ 31. 32.5]
[ 28. 44.5]
[ 30. 58.5]
[ 31. 32.5]
[ 30. 17.5]
[ 29. 16.5]
[ 24. 9.5]
[ 22. 30.5]
[ 26. 25.5]
[ 28. 19.5]
[ 31. 16.5]
[ 34. 22.5]
[ 35. 29.5]]
[[ 23. 52.5]
[ 28. 42.5]
[ 31. 32.5]
[ 31. 32.5]
[ 28. 43.5]
[ 30. 58.5]
[ 31. 32.5]
[ 30. 17.5]
[ 29. 16.5]
[ 25. 9.5]
[ 22. 30.5]
[ 27. 25.5]
[ 28. 18.5]
[ 31. 17.5]
[ 34. 22.5]
[ 35. 29.5]]
[[ 23. 52.5]
[ 28. 42.5]
[ 32. 31.5]
[ 31. 31.5]
[ 28. 43.5]
[ 30. 58.5]
[ 31. 31.5]
[ 29. 17.5]
[ 29. 16.5]
[ 25. 8.5]
[ 22. 30.5]
[ 26. 25.5]
[ 28. 18.5]
[ 31. 17.5]
[ 34. 22.5]
[ 34. 28.5]]
[[ 31. 57.5]
[ 29. 42.5]
[ 32. 31.5]
[ 31. 31.5]
[ 28. 43.5]
[ 30. 58.5]
[ 32. 31.5]
[ 29. 17.5]
[ 28. 16.5]
[ 25. 8.5]
[ 22. 30.5]
[ 27. 25.5]
[ 28. 18.5]
[ 31. 17.5]
[ 34. 22.5]
[ 34. 29.5]]
[[ 24. 53.5]
[ 28. 43.5]
[ 31. 31.5]
[ 31. 31.5]
[ 28. 43.5]
[ 30. 58.5]
[ 31. 31.5]
[ 30. 17.5]
[ 29. 15.5]
[ 25. 7.5]
[ 23. 30.5]
[ 27. 25.5]
[ 29. 17.5]
[ 30. 17.5]
[ 33. 23.5]
[ 34. 28.5]]
[[ 25. 54.5]
[ 28. 43.5]
[ 30. 31.5]
[ 31. 31.5]
[ 28. 43.5]
[ 31. 58.5]
[ 30. 31.5]
[ 29. 17.5]
[ 28. 15.5]
[ 25. 7.5]
[ 24. 29.5]
[ 28. 25.5]
[ 29. 17.5]
[ 30. 17.5]
[ 33. 22.5]
[ 33. 29.5]]
[[ 27. 55.5]
[ 28. 43.5]
[ 30. 30.5]
[ 30. 30.5]
[ 27. 42.5]
[ 30. 57.5]
[ 30. 30.5]
[ 30. 16.5]
[ 29. 15.5]
[ 24. 7.5]
[ 25. 29.5]
[ 28. 24.5]
[ 29. 16.5]
[ 30. 16.5]
[ 32. 23.5]
[ 33. 29.5]]
[[ 28. 55.5]
[ 28. 43.5]
[ 30. 30.5]
[ 30. 30.5]
[ 27. 43.5]
[ 29. 57.5]
[ 30. 30.5]
[ 30. 16.5]
[ 29. 14.5]
[ 25. 7.5]
[ 25. 29.5]
[ 29. 24.5]
[ 30. 16.5]
[ 30. 17.5]
[ 31. 24.5]
[ 32. 30.5]]
[[ 30. 56.5]
[ 28. 42.5]
[ 31. 29.5]
[ 30. 29.5]
[ 27. 42.5]
[ 28. 56.5]
[ 31. 29.5]
[ 30. 16.5]
[ 29. 14.5]
[ 25. 7.5]
[ 25. 29.5]
[ 29. 24.5]
[ 29. 16.5]
[ 30. 15.5]
[ 32. 23.5]
[ 32. 31.5]]
[[ 29. 56.5]
[ 29. 42.5]
[ 31. 29.5]
[ 30. 29.5]
[ 28. 42.5]
[ 29. 56.5]
[ 31. 29.5]
[ 30. 16.5]
[ 29. 14.5]
[ 25. 7.5]
[ 26. 29.5]
[ 29. 24.5]
[ 29. 16.5]
[ 30. 16.5]
[ 31. 24.5]
[ 31. 31.5]]
[[ 30. 57.5]
[ 29. 42.5]
[ 30. 30.5]
[ 30. 30.5]
[ 28. 42.5]
[ 29. 57.5]
[ 30. 30.5]
[ 30. 16.5]
[ 29. 14.5]
[ 25. 7.5]
[ 28. 30.5]
[ 31. 23.5]
[ 30. 16.5]
[ 30. 16.5]
[ 31. 24.5]
[ 28. 30.5]]
[[ 29. 57.5]
[ 29. 43.5]
[ 31. 30.5]
[ 31. 30.5]
[ 28. 43.5]
[ 29. 57.5]
[ 31. 30.5]
[ 30. 16.5]
[ 29. 14.5]
[ 25. 7.5]
[ 28. 30.5]
[ 31. 24.5]
[ 30. 16.5]
[ 30. 16.5]
[ 31. 24.5]
[ 27. 30.5]]
[[ 29. 57.5]
[ 28. 43.5]
[ 30. 31.5]
[ 30. 31.5]
[ 27. 43.5]
[ 29. 58.5]
[ 30. 31.5]
[ 30. 16.5]
[ 29. 14.5]
[ 25. 7.5]
[ 28. 30.5]
[ 31. 24.5]
[ 30. 16.5]
[ 30. 16.5]
[ 31. 24.5]
[ 27. 30.5]]]
(13, 16, 2)
Traceback (most recent call last):
File "create_movie.py", line 240, in <module>
tf.app.run()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "create_movie.py", line 237, in main
create_movie()
File "create_movie.py", line 145, in create_movie
enc_in = enc_in[ :, dim_to_use_2d ]
IndexError: index 16 is out of bounds for axis 1 with size 16