[HELP] Unable to Maintain connectivity between IoT Gateway & Thingsboard CE server/Thingsboard Edge

[sanjayN4497]

I've been attempting to establish a connection between Thingsboard IoT Gateway and Thingsboard CE/Thingsboard Edge. Initially, I tried using the MQTT demo connector method, which worked well. However, after dynamic IP changes occur, the connection fails, resulting in a "timed out" error, even after updating the IP address in the Docker Compose file same in both connectivity of IoT Gateway either with Thingsboard CE or Thingsboard Edge.

Let me share details about the issue below:

1. Here's the setup I've implemented:
   - ThingsBoard server is deployed on AWS EC Server 1.
   - ThingsBoard Edge & ThingsBoard IoT Gateway are installed on AWS EC Server 2.

2. Due to AWS's dynamic IP setup, the IP address changes daily. Initially, during the first setup, my IoT gateway device Docker Compose file looked like the following, and it successfully communicated with Thingsboard Server / Thingsboard Edge:

Docker-Compose.yml for gateway

version: '3.4'
services:
  # ThingsBoard IoT Gateway Service Configuration
  tb-gateway:
    image: thingsboard/tb-gateway
    container_name: tb-gateway
    restart: always

    # Ports bindings - required by some connectors
    ports:
        - "5000:5000" # Comment if you don't use REST connector and change if you use another port
        # Uncomment and modify the following ports based on connector usage:
#        - "1052:1052" # BACnet connector
#        - "5026:5026" # Modbus TCP connector (Modbus Slave)
#        - "50000:50000/tcp" # Socket connector with type TCP
#        - "50000:50000/udp" # Socket connector with type UDP

    # Necessary mapping for Linux
    extra_hosts:
      - "host.docker.internal:host-gateway"

    # Environment variables
    environment:
      - host=xx.yy.zz.aaa #masked for security
      - port=1883
      - accessToken=rwPcmg54xbRr0syBM503

    # Volumes bind
    volumes:
      - tb-gw-config:/thingsboard_gateway/config
      - tb-gw-logs:/thingsboard_gateway/logs
      - tb-gw-extensions:/thingsboard_gateway/extensions

# Volumes declaration for configurations, extensions and configuration
volumes:
  tb-gw-config:
    name: tb-gw-config
  tb-gw-logs:
    name: tb-gw-logs
  tb-gw-extensions:
    name: tb-gw-extensions

3. However, the very next day, the IP changed, so I manually updated the IP in my Docker Compose file accordingly.

Error Encountered:

The IoT gateway encountered the following error after updating the IP in docker compose file and run sudo docker-compose up:

tb-gateway | 2024-03-29 04:31:18 - |ERROR| - [tb_client.py] - tb_client - connect - 270 - timed out
tb-gateway | Traceback (most recent call last):
tb-gateway | File "/thingsboard_gateway/gateway/tb_client.py", line 265, in connect
tb-gateway | self.client.connect(keepalive=keep_alive,
tb-gateway | File "/usr/local/lib/python3.11/site-packages/tb_device_mqtt.py", line 451, in connect
tb-gateway | self._client.connect(self.__host, self.__port, keepalive=keepalive)
tb-gateway | File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 915, in connect
tb-gateway | return self.reconnect()
tb-gateway | ^^^^^^^^^^^^^^^^
tb-gateway | File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 1057, in reconnect
tb-gateway | sock = self._create_socket_connection()
tb-gateway | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway | File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 3731, in _create_socket_connection
tb-gateway | return socket.create_connection(addr, timeout=self._connect_timeout, source_address=source)
tb-gateway | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tb-gateway | File "/usr/local/lib/python3.11/socket.py", line 851, in create_connection
tb-gateway | raise exceptions[0]
tb-gateway | File "/usr/local/lib/python3.11/socket.py", line 836, in create_connection
tb-gateway | sock.connect(sa)
tb-gateway | TimeoutError: timed out

Troubleshooting Steps Taken:

1. Attempted to resolve the issue by disabling ufw using sudo ufw disable #1303 , as suggested in another ticket, but it didn't work.
2. Tried removing the docker image and other troubleshooting methods but none were successful.
3. Noticed that creating a new gateway device or deleting connector devices sometimes resolves the issue, but this is inconsistent and not a reliable solution.

Request for Assistance:

I need assistance in resolving this issue permanently as the current workaround is not reliable. Any guidance or suggestions would be greatly appreciated.

Environment :

OS: Ubuntu on EC2 Instance
IoT Gateway : 3.4.6
ThingsBoard Server: 3.6.3
Thingsboard Edge: 3.6.3
Browser: Chrome

[sanjayN4497]

One more observation: I have a suspicion regarding the dynamic IP aspect of EC2. For debugging purposes, I assigned a static IP yesterday, but the same issue persists. Today, I attempted to resolve the timeout issue through firewall commands, removing connector devices, and deleting images, but none of these efforts resolved it.

Interestingly, even with the static IP assigned, the problem persisted. However, I noticed that creating a new IoT gateway device and using its access token in the Docker Compose file resolves the issue temporarily. Despite this temporary workaround, the problem remains unresolved.

version: '3.4'
services:
  # ThingsBoard IoT Gateway Service Configuration
  tb-gateway:
    image: thingsboard/tb-gateway
    container_name: tb-gateway
    restart: always

    # Ports bindings - required by some connectors
    ports:
        - "5000:5000" # Comment if you don't use REST connector and change if you use another port
        # Uncomment and modify the following ports based on connector usage:
#        - "1052:1052" # BACnet connector
#        - "5026:5026" # Modbus TCP connector (Modbus Slave)
#        - "50000:50000/tcp" # Socket connector with type TCP
#        - "50000:50000/udp" # Socket connector with type UDP

    # Necessary mapping for Linux
    extra_hosts:
      - "host.docker.internal:host-gateway"

    # Environment variables
    environment:
      - host=65.0.98.19
      - port=1883
      - accessToken=lx6ptnyhw1aesegsajva  #New IoT Gateway device Token

    # Volumes bind
    volumes:
      - tb-gw-config:/thingsboard_gateway/config
      - tb-gw-logs:/thingsboard_gateway/logs
      - tb-gw-extensions:/thingsboard_gateway/extensions

# Volumes declaration for configurations, extensions and configuration
volumes:
  tb-gw-config:
    name: tb-gw-config
  tb-gw-logs:
    name: tb-gw-logs
  tb-gw-extensions:
    name: tb-gw-extensions

After addition of new IoT Gateway Device access token in docker-compose.yml file

But it's not feasible to daily create New IoT Gateway Device. Could you please advise on a more permanent solution to this issue? Your guidance or suggestions would be greatly appreciated.