florianblume / 6d-pat

6D - Pose Annotation Tool (6D-PAT) - is a tool that allows the user to load a set of images and also a set of 3D models and annotate where in the 2D image the 3D object ist placed.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AppImage still not working

tobiapoppi opened this issue · comments

Hi florian, i'm sorry to be annoying. It's not a big deal because i installed 6dpat building it from source 2 months ago. Now that i need it on another computer i decided to retry with the AppImage, because i read on the issue you closed that the problem was solved.

Originally posted by @florianblume in #121 (comment)

When i run the appImage on my Ubuntu 20.04 the result is:

"./6DPAT-x86_64.AppImage: error while loading shared libraries: libopencv_calib3d.so.4.3: cannot open shared object file: No such file or directory"

I tried with manual installation of libopencv but nothing changed.
Thanks for your work.

Hey @tobi1modna, thanks for your feedback! That's good to know. I'll investigate it and see why that happens. By accident I discovered another issue with the AppImage - I'm building the app using Qt 5.15.2 in the GitHub actions but that Qt version is missing libraries. I changed it now to Qt 5.14.2 and the next release will hopefully work. In the meantime I suggest you build again from master (maybe in a new folder to keep the old working versions), I fixed some bugs and added some features :)

By the way: Are there any other features that you are really missing in the program? I'm always looking for new things to implement ;)

Thank you so much Florian! I'll follow your suggestion :)
The only thing that for now is coming into my mind is the interpretation of the program of the 'camera info' file.
Having an image to annotate, i look into the details of the file. With focal length and F number i can calculate the fovX and fovY that i give to your python script, to generate the camera matrix json file.
The problem is sometimes i really struggle understanding how the software interpret the 3D space of the annotation, because he puts the annotated model outside the image, maybe because all values X and Y are flipped in the opposite way. Anyway today i'm going to my Lab to take some new photos to better interpret fovX and fovY, having all the same.

All this screed just to say that maybe a bit of wiki on how the program interpret the 3D space, starting from the camera matrix info, would be useful.

Last thing: in the panel with 3D models, it would be useful having more control about the rotation of the model before starting annotating it. The rotation that is now implemented it's ok, but very difficult for me sometimes to control the rotation how i want.
To make it simple maybe just an x y z manual wheel would be enough.

Have a nice weekend! :)

Ah ok, I see. Thanks again for the feedback. You mean you struggle with what I'm expecting as the camera matrix, i.e. in what format? And do you use the default JSON loader or did you write your own Python script that supplies it to the program? You could check out the Python script in the scripts folder called tless.py which is able to load the T-Less dataset. Maybe that makes it more clear. And did you see this wiki page? I updated it lately to make everything more clear. This part
{ "image_filename": { "K": [fx, 0, cx, 0, fx, cx, 0, 0, 1] } } I thought is universal to all camera matrices.

Regarding you other suggestion: I have plans to imlement a 3D gizmo, which I will also add to the pose editor :) I know the rotation is a bit wonky, it's Qt3D's own camera controller and I think the normal way how to rotate objects (I re-implemented it for the pose viewer on the left). I'm not sure when I'll get round to implementing such a complex thing as gizmos but I'll try my best!

Ah or do you mean by "how the program interprets the 3D space" as in what direction the camera looks (along z axis), etc? x is left-right, y is up-down and z is forward-backward, I think that's the same for all OpenGL applications. Or if I misunderstood you correct me, your also welcome to enter parts in the wiki when you feel they don't explain it well.

Hi @florianblume, i hope you are enjoying the summer.
Yesterday i tried to update 6dpat to the new release but i'm so confused and stuck.
I work on kubuntu 20.04, I have qt 5.14 installed, opencv 4.5, python 3.8, pybind11, openGL 4.6 (i tried to downgrade to 3.1 but seems impossible).

With Docker:
In my case Docker is apparently the easiest and the most stable way to get the program running, but still i can't use it.
I run the command 'xhost +local:root'.
Then i search the nvidia files in the "dev" directory in order to mount them, with the command: 'ls -la /dev | grep nvidia'.
My output is:


tobi@tobi-desktop:~$ ls -la /dev | grep nvidia
crw-rw-rw-   1 root root    195,   0 ago 23 13:18 nvidia0
crw-rw-rw-   1 root root    195, 255 ago 23 13:18 nvidiactl
crw-rw-rw-   1 root root    195, 254 ago 23 13:18 nvidia-modeset
crw-rw-rw-   1 root root    234,   0 ago 23 13:18 nvidia-uvm
crw-rw-rw-   1 root root    234,   1 ago 23 13:18 nvidia-uvm-tools

so i run your command:

sudo docker run -ti --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiaclt --device /dev/nvidia-modeset:/dev/nvidia-modeset --device /dev/nvidia-uvm:/dev/nvidia-uvm --device /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools florianblume/6dpat /6DPAT

The window of 6dpat opens (with nothing on it) and on the terminal there is continuously the error:

[unknown] (warning): composeAndFlush: makeCurrent() failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): composeAndFlush: makeCurrent() failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): composeAndFlush: makeCurrent() failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed
[unknown] (warning): Failed to make context current: OpenGL resources will not be destroyed

With AppImage:
the error is:
"./6DPAT-x86_64.AppImage: error while loading shared libraries: libopencv_calib3d.so.4.3: cannot open shared object file: No such file or directory"

So i searched for that library but i can't install it.

Build from source with QtCreator:
The application builds with no errors and runs correctly, but the images can't be visualized. There is a black square on the left, and on the terminal there are a lot of error like this:

Fragment info
-------------
0(3) : error C7555: 'varying' is deprecated, use 'in/out' instead
0(12) : error C7533: global variable gl_FragColor is deprecated after version 120
(0) : error C2003: incompatible options for link

[unknown] (warning): QOpenGLShaderProgram::uniformLocation(matrix): shader program is not linked
[unknown] (warning): QOpenGLShader::link: Vertex info
-----------
0(2) : error C7555: 'attribute' is deprecated, use 'in/out' instead
0(3) : error C7555: 'attribute' is deprecated, use 'in/out' instead
0(4) : error C7555: 'varying' is deprecated, use 'in/out' instead
(0) : error C2003: incompatible options for link

Fragment info
-------------
0(3) : error C7555: 'varying' is deprecated, use 'in/out' instead
0(12) : error C7533: global variable gl_FragColor is deprecated after version 120
(0) : error C2003: incompatible options for link

[unknown] (warning): QOpenGLShaderProgram::uniformLocation(matrix): shader program is not linked

With some researches i thought that this problem was concerned to the OpenGL version (4.6) in which maybe something is deprecated compared to the 3.1 version.
Anyway, no one of my tries had success.

I hope that i'll get to a solution, with at least one of the three methods.
If i get some ideas/evolutions i will update this issue, but i'm so noob into these problems and I'm sure that you have ideas clearer than me. :)

Update!
I was suspecting that the problem was related with my OpenGL version.

My GPU is an NVIDIA geforce GTX 1060, and all nvidia drivers available in my case works with OpenGL 4.6
I tried a lot of different drivers, but all based on 4.6 OpenGL. So i decided to try remove all NVIDIA packages with the command:
sudo apt-get remove --purge '^nvidia-.*'

Then installed Mesa, and fortunately it works with OpenGL 4.3. This solved all the problems with Docker and Building from source with QtCreator.

The AppImage still gives the same error, because that library is not concerned about OpenGL.
But I'm so happy, now i can work with the software!

Hi Tobi,

Sorry to hear you had such trouble. The issue with the appimage is that I accidentally used the wrong qt version in the GitHub action but it worked locally on my computer. In this qt version the libraries are simply missing. The action is fixed and future releases should work properly.

The other issue you had with the graphics driver is weird. I'm not using the keyword 'varying' anywhere and the glsl code should comply to 4.6's standard. I'll investigate this some time. Sorry that you had such trouble.

Btw: I'm nearly finished with the Gizmo implementation. I'll port it to 6D-PAT as soon as it's ready!

And you might have guessed it already I'm currently on vacation so updates will take some time ;) but seems like you figured things out already so I hope you can work with the program we it is for the moment

Yes! I guessed it! I figured it out and I already worked for my thesis! You will be on my references, maybe when it'll be ready I can send it to you 😃.
Have a nice vacation!

I fixed the bug from the other issue you mentioned and created a new release. As long as you have OpenCV 4.2 installed, the AppImage should work now. Could you test that maybe? Unfortunately, somehow the OpenCV libraries are still not included correctly, I'll have to figure this out some time...