KrystianD / modbus_client

Hello @KrystianD,

I was reviewing the code and I saw that in the file pymodbus_async_modbus_client.py the underlying pymodbus clients that are used are the synchronous ones:

modbus_client/modbus_client/pymodbus_async_modbus_client.py

Lines 90 to 99 in 10f79e5

    
           class PyAsyncModbusTcpClient(PyAsyncModbusClient): 
        
               def __init__(self, host: str, port: int, timeout: int): 
        
                   super().__init__(pymodbus.client.tcp.ModbusTcpClient(host=host, port=port, timeout=timeout)) 
        
           class PyAsyncModbusRtuClient(PyAsyncModbusClient): 
        
               def __init__(self, path: str, baudrate: int = 9600, stopbits: int = 1, parity: str = "N", timeout: int = 3): 
        
                   super().__init__(pymodbus.client.serial.ModbusSerialClient(method="rtu", port=path, 
        
                                                                              baudrate=baudrate, stopbits=stopbits, parity=parity, 
        
                                                                              timeout=timeout))

Thus, the pymodbus read and write functions that are called are not asynchronous, this means that for example when making a read operation the pymodbus code will send the request and block until receiving the response. For example, for the TCP client, the code that is finally called in the pymodbus synchronous TCP client to receive the data is the ModbusTcpClient.recv method, which is blocking, these are the relevant lines:

https://github.com/riptideio/pymodbus/blob/94044f134014f90a1144809d6e93ab8a8606a8e5/pymodbus/client/tcp.py#L260-L286

        data = []
        data_length = 0
        time_ = time.time()
        end = time_ + timeout
        while recv_size > 0:
            try:
                ready = select.select([self.socket], [], [], end - time_)
            except ValueError:
                return self._handle_abrupt_socket_close(size, data, time.time() - time_)
            if ready[0]:
                if (recv_data := self.socket.recv(recv_size)) == b"":
                    return self._handle_abrupt_socket_close(
                        size, data, time.time() - time_
                    )
                data.append(recv_data)
                data_length += len(recv_data)
            time_ = time.time()


            # If size isn"t specified continue to read until timeout expires.
            if size:
                recv_size = size - data_length


            # Timeout is reduced also if some data has been received in order
            # to avoid infinite loops when there isn"t an expected response
            # size and the slave sends noisy data continuously.
            if time_ > end:
                break

The code uses the select operation to block until all data is received on the socket or the total time reaches the specified timeout, thus no other operations can be done while waiting for the response.

The solution to this is really simple, the pymodbus asynchronous clients should be used. The pymodbus asynchronous clients use instead a class which inherits from asyncio.Protocol (https://github.com/riptideio/pymodbus/blob/94044f134014f90a1144809d6e93ab8a8606a8e5/pymodbus/client/base.py#L262). The Python asyncio.Protocol class is the way to implement network protocols with asyncio in Python, see https://docs.python.org/3.11/library/asyncio-protocol.html#protocols.

In pymodbus, the asynchronous clients have the protocol and the use_protocol attributes, where use_protocol is just a boolean flag that indicates if the attribute protocol exists. When a read or write request is made in pymodbus one of the first things executed is the ModbusBaseClient.execute method:

https://github.com/riptideio/pymodbus/blob/94044f134014f90a1144809d6e93ab8a8606a8e5/pymodbus/client/base.py#L174-L187

    def execute(self, request: ModbusRequest = None) -> ModbusResponse:
        """Execute request and get response (call **sync/async**).

        :param request: The request to process
        :returns: The result of the request execution
        :raises ConnectionException: Check exception text.
        """
        if self.use_protocol:
            if not self.protocol:
                raise ConnectionException(f"Not connected[{str(self)}]")
            return self.protocol.execute(request)
        if not self.connect():
            raise ConnectionException(f"Failed to connect[{str(self)}]")
        return self.transaction.execute(request)

Here the code checks if use_protocol is true, if so then that means that the client has an asyncio.Protocol in the protocol attribute and will execute the network operations asynchronously.

So, in the pymodbus_async_modbus_client.py I think the changes should be:

Use the asynchronous pymodbus clients, the PyAsyncModbusTcpClient and PyAsyncModbusRtuClient should be as follow:

class PyAsyncModbusTcpClient(PyAsyncModbusClient):
    def __init__(self, host: str, port: int = 502, **kwargs: Any) -> None:
        super().__init__(pymodbus.client.tcp.AsyncModbusTcpClient(
            host=host, port=port, **kwargs))


class PyAsyncModbusRtuClient(PyAsyncModbusClient):
    def __init__(self, path: str, baudrate: int = 9600, stopbits: int = 1, parity: str = "N", **kwargs: Any):
        super().__init__(pymodbus.client.serial.AsyncModbusSerialClient(
            port=path, baudrate=baudrate, stopbits=stopbits, parity=parity, **kwargs))

In the PyAsyncModbusClient class, the constructor can check is the pymodbus client is actually asynchronous:

    def __init__(self, client: pymodbus.client.base.ModbusBaseClient):
        if not client.use_protocol:
            raise ValueError("client must be a pymodbus asynchronous client object")
        self.client = client

Also, in the current code a ThreadPoolExecutor is used for running the pymodbus calls, I am not sure why this was used, but I think it is not needed.

In the PyAsyncModbusClient class, the calls to pymodbus should be made directly with await, for example the write_coil method shoud be as follows:

    async def write_coil(self, slave: int, address: int, value: bool) -> None:
        await self.client.write_coil(slave=slave, address=address, value=value)

As a side note, pymodbus is quite complicated, i just checked this week the details about its inner working. To check all the calls that were made I used the viztracer tool (https://github.com/gaogaotiantian/viztracer), which generates a trace of all the program calls which can be later viewed in the browser with the vizviewer tool, it is very nice.

Later or maybe tomorrow I will make a commit on my fork with the changes.

Hi @tombolano, thank for the issue report.

I am big fan of asynchronous programming and the first thing I tried by the time I was working on this library was to try to use asynchronous version. It was quite some time ago, but as far as I remember I hit some issues with the async version and decided to keep sync version as it was just working well.

The code uses the select operation to block until all data is received on the socket or the total time reaches the specified timeout, thus no other operations can be done while waiting for the response.

This is not true in current version, exactly due to using ThreadPoolExecutor to perform the sync operation in another thread (it is IO blocking, so even with GIL Python limitations it is perfectly fine to offload to separate thread, to make it truly asynchronous). This way, using _run wrapper, I have async version of sync code at almost no cost.

If pymodbus 3.x.x improves asyncio support, I will for sure take a look into it more.

@tombolano yeah, asyncio support in PyModbus is much better now, I implemented the initial version here -> https://github.com/KrystianD/modbus_client/tree/dev-async

Sadly, there is no async common interface, instead they overload sync methods with async counterparts..

I just need to battle test this in my systems, though.

Thanks @KrystianD

Yes, it seems the pymodbus 3.x.x async interface works well, I had tested the changes that I showed in the first message and they seemed to work correctly.

Sadly, there is no async common interface, instead they overload sync methods with async counterparts..

Yes, in this regard the pymodbus interface is weird, this is why in the code I showed before I checked the client.use_protocolattribute, to make sure that the client was asynchronous.

Yes, in this regard the pymodbus interface is weird, this is why in the code I showed before I checked the client.use_protocolattribute, to make sure that the client was asynchronous.

In general, my AsyncModbusClient interface is not to be implemented by the library user (not exported), so creation of the final classes is fixed to PyAsyncModbusTcpClient and PyAsyncModbusRtuClient, but added the check, just in case :)

	class PyAsyncModbusTcpClient(PyAsyncModbusClient):
	def __init__(self, host: str, port: int, timeout: int):
	super().__init__(pymodbus.client.tcp.ModbusTcpClient(host=host, port=port, timeout=timeout))


	class PyAsyncModbusRtuClient(PyAsyncModbusClient):
	def __init__(self, path: str, baudrate: int = 9600, stopbits: int = 1, parity: str = "N", timeout: int = 3):
	super().__init__(pymodbus.client.serial.ModbusSerialClient(method="rtu", port=path,
	baudrate=baudrate, stopbits=stopbits, parity=parity,
	timeout=timeout))

pymodbus async clients should be used