Docker image?
AmitMY opened this issue · comments
Because of the not-trivial build needed here, do you mind creating a dockerfile / a docker image so using this repo is easier?
Thanks for making it available in python!
Thank you for this suggestion @AmitMY
I've never built a Docker image and I don't have time to learn how to do it any time soon.
I'd be happy to accept any contributions though.
So, I need your help here in the custom tensorflow compilation.
I have installed the same python version as I assume you have (from the wheel file it says cc37), as well as checkedout tensorflow tag v1.13.2.
Then I copied the files from mediapipe, and replaced stuff in it like you mention in the README (the sed lines)
I also install the same bazel version you mentioned.
Here is my Dockerfile so far:
FROM python:3.7.1-stretch
RUN apt-get update -y && apt-get upgrade -y
RUN apt-get install sudo git wget build-essential unzip -y
RUN pip3 install numpy opencv-python
RUN git clone https://github.com/wolterlw/hand_tracking.git
RUN git clone https://github.com/google/mediapipe.git
# Clone Tensorflow correct version
RUN git clone https://github.com/tensorflow/tensorflow.git
WORKDIR /tensorflow
RUN git fetch origin tag v1.13.2
RUN git checkout v1.13.2
# Copy the custom module from mediapipe to tensorflow
RUN mkdir /tensorflow/tensorflow/lite/python/custom_ops/
RUN cp -R /mediapipe/mediapipe/util/tflite/operations/. /tensorflow/tensorflow/lite/python/custom_ops/
RUN sed -i 's/mediapipe:__subpackages__/visibility:public/g' /tensorflow/tensorflow/lite/python/custom_ops/BUILD
RUN sed -i 's/@org_tensorflow//g' /tensorflow/tensorflow/lite/python/custom_ops/BUILD
# Install Bazel
RUN wget https://github.com/bazelbuild/bazel/releases/download/0.21.0/bazel-0.21.0-installer-linux-x86_64.sh
RUN chmod +x bazel-0.21.0-installer-linux-x86_64.sh
RUN ./bazel-0.21.0-installer-linux-x86_64.sh
# Bazel Build
RUN bazel build --action_env PATH="$PATH" --noincompatible_strict_action_env --config=opt --incompatible_disable_deprecated_attr_params=false //tensorflow/tools/pip_package:build_pip_package
# Buid pip package
RUN ./bazel-bin/tensorflow/tools/pip_package/build_pip_package
WORKDIR hand_trackingWhen running docker build -t "google-pose" . it fails on the bazel build command.
Config value opt is not defined in any .rc file
Going inside the machine (docker run -it --rm 9dcd2d3463f4 /bin/bash - you see the hash you need to replace instead of 9dcd2d3463f4 in your console) - I run the command without config=opt, which seems to run all of the build stuff, but ends with:
Do you have a clue how I should fix this? You seemed to managed to get this working.
Getting this docker to work once means no one else has to go through this custom build ever again :)
Hmm, the error message does not seem to be anywhere near the custom operations. OK, thanks for such a detailed guide - I'll look into it.
OK I've recreated your issue (although you left out quite a bit of the actual working Dockerfile), but even without custom modifications I could not build tensorflow inside a container.
What do you mean
you left out quite a bit of the actual working Dockerfile
What did I left out?
This Dockerfile is working up to the bazel build line... At least on my machine.
At any rate, thank you for indeed trying to make the build inside the container,
In the upcoming days I'll try further to debug it myself
Had to set environmental variables like TF_NEED_CUDA=0 and stuff.
Sorry I can't be much help now as I caught a terrible flu.
I believe there should be some resources detailing tensorflow installation inside a container. If you manage to do that I believe we'll get this custom build running in no time.
any updates?
Couldn't get it to work yet, but added some commands to the above Dockerfile to get other errors :)
https://github.com/AmitMY/pose-docker/blob/master/Google/Dockerfile
Have you been able to build the Docker Image ?
Thank you :)
Not yet! Didn't have the time to try any more.
The Dockerfile is still the same and you are welcomed to give it a go!
https://github.com/AmitMY/pose-docker/blob/master/Google/Dockerfile
Still, it fails on the bazel build
@wolterlw Hi,
Mind taking a look again in the docker file and trying to fix the error?
@AmitMY
Can you compilate without docker?
Or can you compilate the tflite(official version) in docker?
I don't know how to use docker. But it seems not “custom operations” problem, maybe docker's problem.
Thanks @imLogM , I went through everything again and manage to compile successfully!
Here is the updated Dockerfile anyone can use:
https://github.com/AmitMY/pose-docker/blob/master/Google/Dockerfile
The problem I am facing now is when running:
detector = HandTracker(palm_model_path, landmark_model_path, anchors_path,
box_shift=0.2, box_enlarge=1.3)
kp, box = detector(img)I am getting:
ValueError: Didn't find custom op for name 'Convolution2DTransposeBias' with version 1
I saw you solved a similar problem a while back, so now, with all of my commands in front of you, do you see any problem with the configuration?
@AmitMY
Congratulations! You compile the tflite(official version) successfully.
The problem is because you didn't change the "BUILD file" and "custom operations code files", so you didn't compile "custom operations".
Try to compile using @wolterlw 's method in #2.
In case anybody needs it - here's the full git diff and the BUILD file in the custom_ops directory
https://gist.github.com/wolterlw/767a467d6f75b3533cd49482ba25ae7e
yo guys, wait a sec. I'm uploading a version without a custom tflite build today. Should work like a charm
Cool, like https://github.com/metalwhale/hand_tracking ?
Will your port be able to use the GPU? Would it be able to detect multiple hands?
Looking forward to it.
Especially as after adding the diff using these commands:
sed -i '39i cc_library(name = "common", srcs = [], hdrs = ["common.h"], copts = tflite_copts(), deps = [":cpu_check",":types","@gemmlowp//:fixedpoint"])' /tensorflow/tensorflow/lite/kernels/internal/BUILD
sed -i '19i "//tensorflow/lite/python/custom_ops:max_pool_argmax","//tensorflow/lite/python/custom_ops:max_unpooling","//tensorflow/lite/python/custom_ops:transpose_conv_bias",' /tensorflow/tensorflow/lite/python/interpreter_wrapper/BUILD
sed -i '23i #include "tensorflow/lite/python/custom_ops/max_pool_argmax.h"\n#include "tensorflow/lite/python/custom_ops/max_unpooling.h"\n#include "tensorflow/lite/python/custom_ops/transpose_conv_bias.h"' /tensorflow/tensorflow/lite/python/interpreter_wrapper/interpreter_wrapper.cc
sed -i '204i resolver->AddCustom("MaxPoolingWithArgmax2D", tflite_operations::RegisterMaxPoolingWithArgmax2D()); resolver->AddCustom("MaxUnpooling2D", tflite_operations::RegisterMaxUnpooling2D());resolver->AddCustom("Convolution2DTransposeBias", tflite_operations::RegisterConvolution2DTransposeBias());' /tensorflow/tensorflow/lite/python/interpreter_wrapper/interpreter_wrapper.ccI get:
@AmitMY yeah, that's exactly the repo I'm referring to.
Unfortunately no, it can't run on GPU.
Found what looks like the desktop GPU delegate in bazel-mediapipe, but I'm moving on to build my own hand tracking pipeline with emphasis on tracking
Would it be able to detect multiple hands?
Because the other one works only on a single hand
check out optical_flow branch - you'll find hand_tracker2.py which supports multiple hand tracking. However that branch is a mess so far.
But if there's demand - I'll clean it up sometime this week.
If you wanna build a Docker image - you can do it now, the dependencies aren't changing.
Thank you, I will build that docker image now.
One last question - I might be mistaken but I can only see support for 2D hand pose estimation, but the Google library supports 3D afaik -
https://github.com/google/mediapipe/blob/master/mediapipe/docs/hand_tracking_mobile_gpu.md
If that is the case, what is necessary for 3D support?
Getting close!
With the new code, I ran:
# This is copied from the jupyter code
img = cv2.imread('./data/test_img.jpg')[:, :, ::-1]
detector = HandTracker(palm_model_path, landmark_model_path, anchors_path, box_shift=0.2, box_enlarge=1.3)
kp, box = detector(img)and got:
Dockerfile:
FROM tensorflow/tensorflow:latest-py3
RUN apt-get update -y && apt-get upgrade -y
RUN apt-get install sudo wget git libsm6 libxext6 libxrender-dev -y
# Install python dependencies
RUN pip install numpy opencv-python matplotlib
WORKDIR /
RUN git clone https://github.com/wolterlw/hand_tracking.git
WORKDIR /hand_tracking
RUN mkdir models
RUN ./load_models.shYeah, that's alright, I've fixed that in the multihand version, not a very serious issue
Thanks. Sorry if I'm missing something but:
- Is this supporting 3D? (See 3 comments up)
- So the next steps for me are to use
hand_tracker.pyfromoptical_flowbranch?
- Haven't looked into 3D, but it should be quite similar
- Wait for me to merge it into master
You can easily access the 3rd coordinates: ahmed-shariff@310fb95
hope that helps.
Hi, I have one question. The value of anchors of column three and four in anchors.csv is 1, which is different from the anchors of SSD. Do you know how to construct the true boxes (the dimension is [num_priors, 4] like SSD) using anchors.csv to calculate the loss in the training phase? Thank you so much !
Thank you @ahmed-shariff !
If @wolterlw does not incorporate this to the merge it seems easy to add by myself.
@wolterlw please update this thread if you can once you merged into master! Thanks for all of the great work!
@Damilytutu as that is unrelated, please open a new issue
Hi @wolterlw
Not to rush you or anything, but do you have an estimate when will the hand_tracking branch be ready, so we can avoid the overflow error?
(I would just like to know if soon, a few days, or we are talking weeks or months)
check out the multihand branch. It is sorta ready, but some improvements could be made with detection aggregation as the landmark model is not incredibly robust to rotation.
Thanks.
I'll make that docker now when you merge the 3d changes.
3D output is pretty. Thanks @ahmed-shariff
From:
To whom it may concern -
Docker configuration can be found in https://github.com/AmitMY/pose-docker/tree/master/Google
I will probably also push it to Dockerhub in the future.
Instructions are in the README.md.
It supports 2d or 3d hand pose estimation from video.
great job,
I say you should add the links to both your repo and dockerhub into this repo's Readme with all due credits.
@AmitMY Hi Amit! Would you mind explaining how you managed to get the connections between joints? Running the hand_landmark and palm_detection_without_op models only gives me the keypoints. Do you know how I can obtain the connections between joints (complete 2D pose)?
P.S. I am using the mutihand branch
You can find a definition of HAND_POINTS and HAND_LIMBS here - https://github.com/AmitMY/pose-format/blob/master/pose_format/utils/openpose.py#L48
To get the indexes perhaps you could do
LIMB_INDEXES = [(HAND_POINTS.index(a), HAND_POINTS.index(b) for a, b in HAND_LIMBS]@AmitMY Thanks this is great. But how do you incorporate this in the present code (hand_tracker.py)? I am not really sure where in the code the generated keypoints are assigned to certain joints in the hand - so don't know how I can use your definition and then draw the limb connections. Could you please help me with that?
what do you mean by “assigned”. The ordering of predicted joints is constant, so to draw the index finger, for example, you’d have to draw a line that connects points 5,6,7,8, according to @AmitMY’s repo. Also you can always explore stuff like that in a Jupyter notebook for pose schemes that are new.