Bluetooth hub disconnects during firmware flash on MacOS
legolasse opened this issue · comments
The error is observed on the following setup:
OS: Apple M1 Pro, 14.4.1 (23E224)
Python: 3.10.4
Firmware: https://nightly.link/pybricks/pybricks-micropython/workflows/build/master/technichub-firmware-build-3343-git46f29c09.zip
pybricksdev: v1.0.0a46
Running: pybricksdev flash with the given firmware causes the error:
Creating firmware...
Searching for TECHNIC hub...
Found: DF4BFD20-FC05-F0E9-3AFD-8A9E3DAF5A90: LEGO Bootloader
Connecting to DF4BFD20-FC05-F0E9-3AFD-8A9E3DAF5A90: LEGO Bootloader
Connected successfully!
Erasing flash and starting update
1%|█▏ | 2.21k/200k [00:30<45:56, 71.9B/s]
Traceback (most recent call last):
File "/Users/ld/.pyenv/versions/3.10.4/bin/pybricksdev", line 8, in <module>
sys.exit(main())
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/pybricksdev/cli/__init__.py", line 383, in main
asyncio.run(subparsers.choices[args.tool].tool.run(args))
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/pybricksdev/cli/flash.py", line 472, in flash_firmware
await flash_ble(hub_kind, firmware, metadata)
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/pybricksdev/cli/flash.py", line 327, in flash_ble
await updater.flash(firmware, metadata)
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/pybricksdev/flash.py", line 241, in flash
result = await self.bootloader_request(
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/pybricksdev/flash.py", line 160, in bootloader_request
await self.write(data, request.write_with_response)
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/pybricksdev/ble/__init__.py", line 149, in write
await self.client.write_gatt_char(
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/bleak/__init__.py", line 659, in write_gatt_char
await self._backend.write_gatt_char(char_specifier, data, response)
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/bleak/backends/corebluetooth/client.py", line 332, in write_gatt_char
await self._delegate.write_characteristic(
File "/Users/ld/.pyenv/versions/3.10.4/lib/python3.10/site-packages/bleak/backends/corebluetooth/PeripheralDelegate.py", line 192, in write_characteristic
await future
bleak.exc.BleakError: disconnected
This issue currently makes PyBricks unavailable for the given OS/chip combo due to neither Chromium nor pybricksdev working.
If you have any other Bluetooth devices connected to your Mac, I would turn them all off and try again.
If that doesn't fix it, we would need Bluetooth packet logs to help troubleshoot. (see https://bleak.readthedocs.io/en/latest/troubleshooting.html#macos)
Thank you @legolasse ! I'm moving this here as it affects both Pybricksdev and Pybricks Code via Chrome.
Maybe Apple broke something :) Logs as David mentioned above would be super helpful.
I have the same issue, using Chrome browser on Mac (Apple M1 Pro, 14.4.1 (23E224)) using currrent https://caos.pybricks.com (12.4.2024), MicroPython v3.5.0, Pybricks Code v2.5.0
Any attempt to flash a Technic Hub with the new firmware starts erasing the old firmware and then counting up to 2% or a little higher (It did never get above 9%). After some time the message appears: "The hub took too long to respond. Restart the hub and try again."
I followed the tips on https://github.com/orgs/pybricks/discussions/270. I made the Mac "forget" all bluetooth devices. I disconnected all motors from the hub. Nothing helped.
The packet logger shows an entry "Remote User Terminated Connection" (18:16:25.631) (see Screenshot).
In the log file there is a time shift of two hours, so the relevant event appears to be at 20:16:25.631 CEST.
Technic Hub Disconnect.pklg.zip (I am not experienced with PacketLogger or Wireshark, so I hope the file contains any useful information.)
I have access to another Mac (3,1 GHz Intel i7, 13.6.6 (22G630)). Here it worked (!) using the same https://caos.pybricks.com, MicroPython v3.5.0, Pybricks Code v2.5.0: flashing the Technic Hubs succeeds without problems on that machine.
Just for clarity, I assume you meant code.pybricks.com instade of caos those letters are rather close to each other on the keyboard 😉
Bert
Thanks for the logs. It helps to have our plugin for decoding the packets.
We can see that the last thing before the disconnect was a checksum request that never receive a reply.
So most likely, your Bluetooth on this particular computer is so efficient that it sent data to the hub faster than the hub could process it, causing the hub to lock up.
So we probably need to slow down firmware updates a bit to make sure this can't happen.
Just for clarity, I assume you meant code.pybricks.com instade of caos those letters are rather close to each other on the keyboard 😉
Bert
Oops. Of course, you are right. Sorry for the mistake.
Nicolas
As a test, the following pybricksdev branch adds a bunch of waiting. The firmware install is now intentionally very slow, but it would be interesting if it could now get past the hang.
If you'd like to try it, he's how:
git clone https://github.com/pybricks/pybricksdev.git
cd pybricksdev
pip install poetry # Or install poetry some other way, like with pipx
poetry install
git fetch
git checkout mac-flash
pybricksdev flash path/to/your/downloaded/firmware.zip # Firmware install will now very slow
Thanks!
I can't seem to figure out where to find the firmware to download, so I can't test the method above.
Anyways, I can confirm that this issue has existed at least since before christmas 2023 on my M1 Pro MacBook Pro as well. It will work to update about 5% of the time. 95% of the time it times out before completing. It sure feels like it has to do with the speed since my 2013 Mac works every time, but super slowly.
You can use any firmware version to test it. Here's a convenient link to find the latest: https://nightly.link/pybricks/pybricks-micropython/workflows/build/master
Thanks!
I'm sorry if I'm simply doing something wrong. I get the following. Maybe I should let someone who knows what they are doing do this 😅
File "/usr/local/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/zipfile/init.py", line 1532, in getinfo
raise KeyError(
KeyError: "There is no item named 'main.py' in the archive"
KeyError: "There is no item named 'main.py' in the archive"
This sounds like you might be using an older version of pybricksdev instead of the branch Laurens mentioned.
Also, perhaps another thing to try: if you have any other Bluetooth devices connected to your Mac, disconnect them before using Pybricks. It should speed things up and help avoid problems.
After debugging with laurensvalk, we found the following delay to work in flash.py (around line 243):
if i % 10 == 9:
await asyncio.sleep(0.01) # Any lower than 0.001 causes timeout on some devices. 0.01 is to be on the safe side.
result = await self.bootloader_request(
self.GET_CHECKSUM, timeout=0.5
)
Instead of adding a delay, can we fix it by doing fewer writes in between each checksum request? And compare the throughput on each to see which is more efficient.