Add functions to quickly generate sample pose objects
bricksdont opened this issue · comments
Based on this discussion:
AI4Bharat/OpenHands#29 (comment)
I think there is a need for code to quickly generate pose objects, with the Posebody type as a variable argument. I can take the code for this from unit tests, e.g. https://github.com/AmitMY/pose-format/blob/master/pose_format/pose_test.py#L159
This is what I had in mind:
random_pose_object = pose_format.Pose.random(pose_type="openpose_137", body_tpe="numpy", num_frames=10)
It still gives a feel (atleast to me) as if this .pose format is some blackbox. My suggestion is that it would be best if the users had a feel as if this file is just an efficient wrapper around a 4D tensor of shape (num_persons, num_frames, num_keypoints, num_coordinates)
Something like this might be a bit better:
random_pose_object = pose_format.Pose.random(num_frames=10, num_keypoints=137, num_coordinates=3)Thanks for your comments @GokulNC ! I think that we can provide something like this, yes.
Some explanations for the code I first proposed:
- The issue with keypoints (and "num_coordinates") is that a pose file currently always has a header that describes all the keypoints that the data must contain. I think that is very useful. At the moment we have 2 "real" headers: openpose and holistic.
- If the random data does not match the header type (example: you generate 14 keypoints, but use the openpose header which expects 137), perhaps some code breaks.
- Similarly for the "body_type", internally the array of actual data can be stored differently (as numpy, tensorflow or torch). There can be defaults for this (e.g. default to numpy) but the complexity cannot completely be hidden from the user. Example: a tensorflow preprocessing pipeline would not allow data arrays in numpy format.
Interesting. But why is it mandatory to have headers for model_names like openpose or holistic?
What if I want to store a manually annotated keypoints dataset? (like coco)
Or a video directly recorded from Microsoft Kinect?
Isn't better to allow it to be generic? Especially since there are many more pose detectors out there and covering all cases might be very difficult / unnecessary.
Regarding body_type, looking at this binary spec, I thought the body was generic & standardized, and all the notions of high-level framework formats like numpy/tf/torch is for user's loading convenience.
(we need to wait for @AmitMY for definitive information, since he is the main developer of this library :) )
My personal opinion is that headers lead to more data integrity and lower the chances of introducing bugs inadvertently.
Regarding the spec: the way a pose file is stored on disk is indeed generic and independent of numpy/tf/torch. But once it is loaded into memory, it holds the data as either numpy, tf or torch.
If you'd like to propose a different solution we'd love to hear about it, and are open to contributions as well.
@GokulNC The format is generic. You can create a COCO header, or a Kinect header, or whatever you want, it is quite simple I think.
What @bricksdont is suggesting here iirc, is a for-testing-only utility function to create an OpenPose or a holistic pose, including the header and a sample body (that puts the correct keypoints at the correct indexes)
The body is also generic - but to have it in memory you have to make a decision how to load it - with numpy? torch? tensorflow? a custom implementation?
Would you like to maybe have a call discussing this?
OK, this makes sense. Thanks!
I will spend an hour or two looking at this full repo code in detail soon when I find time and I'll let you know if I have any doubts/clarifications to discuss about :)
(Sorry for diverging from what this issue was originally about)