Issues during reappearance

Question

Issues during reappearance

JaceZhang1020 opened this issue 2 months ago · comments

Dear aadhithya14,

Greetings!

I am a user from China who is currently attempting to replicate the Simulation content outlined in the "Robot Controller Installation Specific Information" from your project. I am using a Quest 3 device with an Ubuntu 20.04 server system.

During my attempt to run the command python3 teleop.py robot=libero_sim sim_env=True, I encountered several issues. The specific error messages are as follows:

(openteach) ➜  Open-Teach git:(main) ✗ python3 teleop.py robot=libero_sim sim_env=True
***************************************************************
     Starting keypoint position transform component
***************************************************************
***************************************************************
     Starting vr detector component
***************************************************************
Process Process-2:
Process Process-3:
Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
    return _target_(*args, **kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/detector/oculusbimanual.py", line 18, in __init__
    self.raw_keypoint_right_socket = create_pull_socket(host, oculus_right_port)
  File "/home/ghost/workbench/Open-Teach/openteach/utils/network.py", line 20, in create_pull_socket
    socket.bind('tcp://{}:{}'.format(host, port))
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/zmq/sugar/socket.py", line 302, in bind
    super().bind(addr)
  File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Cannot assign requested address (addr='tcp://172.24.71.206:8087')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/initializers.py", line 112, in _start_component
    component = hydra.utils.instantiate(configs)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 222, in instantiate
    return instantiate_node(
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 339, in instantiate_node
    return _call_target(_target_, partial, args, kwargs, full_key)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'openteach.components.detector.oculusbimanual.OculusVRTwoHandDetector':
ZMQError('Cannot assign requested address (addr='tcp://172.24.71.206:8087')')
full_key: robot.detector
Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
    return _target_(*args, **kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/detector/keypoint_transform.py", line 16, in __init__
    self.transformed_keypoint_publisher = ZMQKeypointPublisher(host, transformation_port)
  File "/home/ghost/workbench/Open-Teach/openteach/utils/network.py", line 39, in __init__
    self._init_publisher()
  File "/home/ghost/workbench/Open-Teach/openteach/utils/network.py", line 44, in _init_publisher
    self.socket.bind('tcp://{}:{}'.format(self._host, self._port))
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/zmq/sugar/socket.py", line 302, in bind
    super().bind(addr)
  File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Cannot assign requested address (addr='tcp://172.24.71.206:8089')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/initializers.py", line 112, in _start_component
    component = hydra.utils.instantiate(configs)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 222, in instantiate
    return instantiate_node(
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 339, in instantiate_node
    return _call_target(_target_, partial, args, kwargs, full_key)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'openteach.components.detector.keypoint_transform.TransformHandPositionCoords':
ZMQError('Cannot assign requested address (addr='tcp://172.24.71.206:8089')')
full_key: robot.transforms0
Process Process-1:
Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/utils.py", line 639, in _locate
    obj = getattr(obj, part)
AttributeError: module 'openteach.components.environment' has no attribute 'libero_env'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/utils.py", line 645, in _locate
    obj = import_module(mod)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ghost/workbench/Open-Teach/openteach/components/environment/libero_env.py", line 10, in <module>
    import robosuite.utils.transform_utils as T
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/__init__.py", line 1, in <module>
    from robosuite.environments.base import make
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/environments/__init__.py", line 1, in <module>
    from .base import REGISTERED_ENVS, MujocoEnv
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/environments/base.py", line 12, in <module>
    from robosuite.utils.binding_utils import MjRenderContextOffscreen, MjSim
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/utils/binding_utils.py", line 58, in <module>
    from robosuite.renderers.context.egl_context import EGLGLContext as GLContext
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/renderers/context/egl_context.py", line 32, in <module>
    from mujoco.egl import egl_ext as EGL
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/mujoco/egl/__init__.py", line 67, in <module>
    raise ImportError(
ImportError: Cannot initialize a EGL device display. This likely means that your EGL driver does not support the PLATFORM_DEVICE extension, which is required for creating a headless rendering context.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 134, in _resolve_target
    target = _locate(target)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/utils.py", line 653, in _locate
    raise ImportError(
ImportError: Error loading 'openteach.components.environment.libero_env.LiberoEnv':
ImportError('Cannot initialize a EGL device display. This likely means that your EGL driver does not support the PLATFORM_DEVICE extension, which is required for creating a headless rendering context.')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/initializers.py", line 112, in _start_component
    component = hydra.utils.instantiate(configs)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 222, in instantiate
    return instantiate_node(
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 325, in instantiate_node
    _target_ = _resolve_target(node.get(_Keys.TARGET), full_key)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 139, in _resolve_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error locating target 'openteach.components.environment.libero_env.LiberoEnv', see chained exception above.
full_key: robot.environment0
***************************************************************
     Starting hand 2D plotter component
***************************************************************
Process Process-4:
Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
    return _target_(*args, **kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/visualizers/visualizer_2d.py", line 16, in __init__
    self.plotter2D = PlotHand2D(host, oculus_feedback_port, display_plot)
  File "/home/ghost/workbench/Open-Teach/openteach/components/visualizers/plotters/plotter_2d.py", line 33, in __init__
    self.socket = ZMQCompressedImageTransmitter(host = host, port = port)
  File "/home/ghost/workbench/Open-Teach/openteach/utils/network.py", line 176, in __init__
    self._init_publisher()
  File "/home/ghost/workbench/Open-Teach/openteach/utils/network.py", line 181, in _init_publisher
    self.socket.bind('tcp://{}:{}'.format(self._host, self._port))
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/zmq/sugar/socket.py", line 302, in bind
    super().bind(addr)
  File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Cannot assign requested address (addr='tcp://172.24.71.206:15001')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/initializers.py", line 112, in _start_component
    component = hydra.utils.instantiate(configs)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 222, in instantiate
    return instantiate_node(
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 339, in instantiate_node
    return _call_target(_target_, partial, args, kwargs, full_key)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'openteach.components.visualizers.visualizer_2d.Hand2DVisualizer':
ZMQError('Cannot assign requested address (addr='tcp://172.24.71.206:15001')')
full_key: robot.visualizers0
Process Process-5:
Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/utils.py", line 639, in _locate
    obj = getattr(obj, part)
AttributeError: module 'openteach.components.operators' has no attribute 'libero_sim'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/utils.py", line 645, in _locate
    obj = import_module(mod)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ghost/workbench/Open-Teach/openteach/components/operators/libero_sim.py", line 17, in <module>
    import robosuite.utils.transform_utils as T
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/__init__.py", line 1, in <module>
    from robosuite.environments.base import make
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/environments/__init__.py", line 1, in <module>
    from .base import REGISTERED_ENVS, MujocoEnv
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/environments/base.py", line 12, in <module>
    from robosuite.utils.binding_utils import MjRenderContextOffscreen, MjSim
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/utils/binding_utils.py", line 58, in <module>
    from robosuite.renderers.context.egl_context import EGLGLContext as GLContext
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/robosuite/renderers/context/egl_context.py", line 32, in <module>
    from mujoco.egl import egl_ext as EGL
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/mujoco/egl/__init__.py", line 67, in <module>
    raise ImportError(
ImportError: Cannot initialize a EGL device display. This likely means that your EGL driver does not support the PLATFORM_DEVICE extension, which is required for creating a headless rendering context.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 134, in _resolve_target
    target = _locate(target)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/utils.py", line 653, in _locate
    raise ImportError(
ImportError: Error loading 'openteach.components.operators.libero_sim.LiberoSimOperator':
ImportError('Cannot initialize a EGL device display. This likely means that your EGL driver does not support the PLATFORM_DEVICE extension, which is required for creating a headless rendering context.')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ghost/workbench/Open-Teach/openteach/components/initializers.py", line 112, in _start_component
    component = hydra.utils.instantiate(configs)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 222, in instantiate
    return instantiate_node(
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 325, in instantiate_node
    _target_ = _resolve_target(node.get(_Keys.TARGET), full_key)
  File "/home/ghost/miniconda3/envs/openteach/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 139, in _resolve_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error locating target 'openteach.components.operators.libero_sim.LiberoSimOperator', see chained exception above.
full_key: robot.operators0
(openteach) ➜  Open-Teach git:(main) ✗

Based on my understanding, these issues may be related to ZMQ address binding and EGL device display initialization. I have tried the suggestions from ChatGPT, but I have not been able to resolve the issues.

For the ZMQ address binding error, I would like to know how to correctly obtain the port numbers for the headset and the two controllers. Are there any specific tools or commands that can assist me in diagnosing and resolving this issue?

Regarding the EGL device display initialization error, I understand that it may be due to the absence of an EGL driver that supports the PLATFORM_DEVICE extension. Could you please advise me on how to install or update this driver so that I can successfully initialize the EGL device display?

I am eagerly awaiting your response and suggestions and hope to resolve these issues as soon as possible to continue with my project research.

Thank you very much for your time and assistance!

Best regards,
JaceZhang1020

Aadhithya Iyer · Answer 1 · Sat Mar 30 2024 02:40:55 GMT+0800 (China Standard Time)

Hi ,
Thanks for raising this issue. As mentioned in the docs , you have to enter your IP of the network. Right now the IP is set at 172.24.71.206. Change this to your machine IP. That should solve the problem of ports. For the problem of EGL drivers, try running the teleoperation using MUJOCO_GL=egl python teleop.py sim_env=True.

JaceZhang1020 · Answer 2 · Wed Apr 03 2024 08:58:51 GMT+0800 (China Standard Time)

Hi,
Thank you for your reply.I wonder if I need to change the IP address of hand controllers as well please,since I also saw them in network.yaml.Meanwhile,I wanna make sure if you mean that I should use command-line parameter MUJOCO_GL=egl sim_env=True while running python teleop.py?

Aadhithya Iyer · Answer 3 · Fri Apr 05 2024 04:56:00 GMT+0800 (China Standard Time)

Hi,
You don't need to change anything except the IP of the network if you are just running the simulation. And to answer the other question yes, try MUJOCO_GL=egl python teleop.py sim_env=True.

Best,
Aadhithya Iyer

JaceZhang1020 · Answer 4 · Sun Apr 07 2024 03:24:59 GMT+0800 (China Standard Time)

Understood.I will try it,thank you again~

Aadhithya Iyer · Answer 5 · Fri Apr 19 2024 03:21:48 GMT+0800 (China Standard Time)

Hi ,
I wanted to check if you were able to get things running on Libero environment. Looking forward to hearing from you.

Best,
Aadhithya Iyer

JaceZhang1020 · Answer 6 · Fri Apr 19 2024 08:35:55 GMT+0800 (China Standard Time)

Hi， Thank you for thinking about this thing all the time.Recently I am busy on my exams so have no time on teleoperation for the moment,so I make no process.If I meet new problem about previous,I will give you feedback ASAP. Thank you again for asking :) Best, JaceZhang1020 获取Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Aadhithya Iyer ***@***.***> Sent: Friday, April 19, 2024 3:22:09 AM To: aadhithya14/Open-Teach ***@***.***> Cc: ***@***.*** ***@***.***>; Author ***@***.***> Subject: Re: [aadhithya14/Open-Teach] Issues during reappearance (Issue #4) Hi , I wanted to check if you were able to get things running on Libero environment. Looking forward to hearing from you. Best, Aadhithya Iyer ― Reply to this email directly, view it on GitHub<#4 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BE2XFCHM4QE6S4QR7FFTKDLY6AMODAVCNFSM6AAAAABFOI62LOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRVGAZTENZWG4>. You are receiving this because you authored the thread.Message ID: ***@***.***>