Invalid protobuf message: expected bytes, bytearray found
BrentVerhaegen opened this issue · comments
Was building a new version of my Docker container that is using the ESPHome API.
Upon building, the following error gets raised. This error wasn't there before, and the changes I made to my container are unrelated to the ESPHome part, so I believe something got broken recently in the API package
energy-meter-1.local: Connection error occurred: Invalid protobuf message: expected bytes, bytearray found
Fatal error: protocol.data_received() call failed.
protocol: <aioesphomeapi._frame_helper.APIPlaintextFrameHelper object at 0x7f6fedd8bb50>
transport: <_SelectorSocketTransport closing fd=6 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 876, in _read_ready__data_received
self._protocol.data_received(data)
File "/usr/local/lib/python3.10/site-packages/aioesphomeapi/_frame_helper.py", line 195, in data_received
self._callback_packet(msg_type_int, packet_data)
File "/usr/local/lib/python3.10/site-packages/aioesphomeapi/_frame_helper.py", line 110, in _callback_packet
self._on_pkt(Packet(type_, data))
File "/usr/local/lib/python3.10/site-packages/aioesphomeapi/connection.py", line 581, in _process_packet
msg.ParseFromString(pkt.data)
TypeError: expected bytes, bytearray found
The ESPHome device is and ESP32 and is running on ESPHome 2022.12.3 FW
Let me know if you need any debugging or more information
Which version of protobuf do you have installed?
Protobuf verion 4.21.12
The one that comes with aioesphomeapi when installing it with pip3
Here are all my packages and their version in the container
aioesphomeapi==13.0.3
async-timeout==4.0.2
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.1.1
cryptography==39.0.0
idna==3.4
ifaddr==0.2.0
noiseprotocol==0.3.1
protobuf==4.21.12
pycparser==2.21
pymodbus==3.0.2
requests==2.28.1
urllib3==1.26.13
zeroconf==0.47.1
I have 4.21.11. Will try upgrading and see if I can repro
No luck there. Works as expected for me.
Tried it with pip3 install --upgrade protobuf --force --no-binary protobuf as well to see if its a difference between the CPP version and pure python version but no error there either
I just tried with older protobuf version, and that indeed doesn't fix the issue that I'm seeing.
But going back to an older version of aioesphomeapi does seem to fix the problem.
aioesphomeapi 13.0.3 seems to be causing this above error, but aioesphomeapi 13.0.2 is working fine
Its going to be a bytearray in 13.0.3 and not bytes but the question is why does ParseFromString reject bytearray for you and not for all the systems I have
Which Python version are you running? The above error was thrown on Python 3.10
I just also tried building my container with Python 3.8, and that doesn't seem to throw the error, but 3.10 does
% python3 --version
Python 3.10.9
Hmm exactly same version on my end..
Could it be that I'm transfering some different data over the API than you are when testing?
Here's a print of my states (got this by running the package with Python 3.8)
Redacted
Does this help you in any way? Can I provide some other data?
I'll adjust the exception to provide more information
Well I wrote a test for this and it passes locally but just failed the CI so some progress
I'd really like to avoid the overhead of converting bytearray to bytes so I'm still working on a fix
I guess we are stuck with the type conversion overhead. I can't find a way around it or figure out why it doesn't affect my systems.
I just pushed 13.0.4 which should fix the issue. Thanks for the report.