ESP32 Reset To Bootloader Issues on Windows (ESPTOOL-386)
negativekelvin opened this issue · comments
On windows, it seems like there are timing issues with the reset to bootloader functions using the DTR and RTS circuit for esp32 because setDTR and setRTS are sent separately. Possible fix:
# issue reset-to-bootloader:
# RTS = either CH_PD or nRESET (both active low = chip in reset)
# DTR = GPIO0 (active low = boot to flasher)
self._port._dtr_state = False
self._port._rts_state = True
self._port._reconfigure_port()
time.sleep(0.05)
self._port._dtr_state = True
self._port._rts_state = False
self._port._reconfigure_port()
time.sleep(0.05)
self._port.setDTR(False)I haven't scoped this or tested it because my v1 board has the transistor bug.
Thanks @negativekelvin. I don't know if you're following the forum thread, but I'm guessing you are.
http://esp32.com/viewtopic.php?f=2&t=334&p=1503#p1497
I considered this solution as well (although I didn't test it either.) The problem is, I'm not comfortable relying on the private interface of pyserial. At minimum, it's necessary to check it hasn't changed over any previous supported pyserial versions. But even if it works with all those, if the private interface changes in the future then it could break esptool without warning.
A hacky but somewhat safer way would be to check all these properties exist with hasattr() before calling them, and fail back to the public API instead. But this is still super hacky.
The safest, most maintainable, fix that I can think of is probably to call the win32 GetCommState/SetCommState API functions directly. This is a stable public API, and the only private part of pyserial it needs to touch is the port handle. I still don't like that idea, but it seems least likely to suddenly break.
Or to submit a patch to pyserial, some variant of the existing apply_settings() that applies all settings as an "atomic" operation.
Yes, that makes sense. I don't know if those properties and methods are very likely to change, so yes I like the check and fall back strategy until pyserial can support it.
However it also looks like _reconfigure_port() only updates the RTS/DTR lines on windows, not other platforms. So would have to add a platform check before calling.
For full details on this issue I have done a full analysis of the problem on Win7 see
http://esp32.com/viewtopic.php?f=2&p=1511#p1511
I also confirm that by modifying esptool.py
from
self._port.setDTR(False)
self._port.setRTS(True)
time.sleep(0.05)
self._port.setDTR(True)
self._port.setRTS(False)
time.sleep(0.05)
self._port.setDTR(False)
to
self._port.setDTR(False) # GPIO0 -> 1
self._port.setRTS(True ) # RST -> 0
self._port.setDTR(True ) # GPIO0 -> 0
time.sleep(0.05)
self._port.setRTS(False ) # RST -> 1
time.sleep(0.1)
That fix the problem on win7 (thanks to rudi post http://esp32.com/viewtopic.php?f=13&p=1512#p1512)
In fact after analysis of rudi patch it is pure luck (thanks to watchdog and glitch on EN signal)
For more details on Rudi patch see Saleae Logic 1.2.10 capture:
- RudiPatch Saleae Capture
- See screenshot:
I have simplified it like that
esptool.py modifications (MyPatch):
from
self._port.setRTS(True)
time.sleep(0.05)
self._port.setDTR(True)
self._port.setRTS(False)
time.sleep(0.05)
self._port.setDTR(False)
to
self._port.setDTR(False) # EN=0 and IO0=0
time.sleep(0.05)
self._port.setRTS(False) # EN=1 and IO0=1(glitch on IO0 during 1ms)
self._port.setDTR(True) # EN=1 and IO0=0
The main problem is the glitch on IO0 during 1.1ms which does not allow to enter download mode as IO0 is sampled by ESP32 during this glitch but we are lucky that we have a reset by watchdog after 0.385s (with message RTCWDT_RTC_RESET) which then boot in download mode because EN=1 and IO0=0
For more details see Saleae Logic 1.2.10 capture of MyPatch:
- MyPatch Saleae Capture
- See screenshot:
For information MyPatch is a bit faster (300ms) compared to Rudi version and keep EN to 0 during at least 50ms(and not 3.5ms like Rudi which in fact is a glitch) and after multiple test my patch never fail with Win7 64bits and also under VirtualBox with Xubuntu 14.04
If other can give feedback it will be great
@bvernoux are you able to capture the waveforms for the method using reconfigure_port in the first post on windows?
I just tried a different fix for this issue (1f5d9d2) which should be equivalent to calling _reconfigure_port, and I thought would definitely correct the issue.
Unfortunately the timing seems to be the same (trace attached), even when both DTR & RTS are set in the same Windows API call. Which I think means the CP2102 Windows driver is doing this, and there's no way to change that pulse in software.
That is disappointing. Time to ask silicon labs?
we are lucky that we have a reset by watchdog after 0.385s
What is the cause of this?
we are lucky that we have a reset by watchdog after 0.385s
What is the cause of this?
It's a silicon bug, that will be fixed at some point.
A solution that seems to work reliably is to add a ~2.2uF capacitor (I think 470nF-2.2uF range should be suitable) to the EN pin (between EN and GND), so it rises slower (there is a ~12K pullup on this pin.) This is not super desirable, as it requires a hardware change, but it works.
I confirm adding a 2.2uF capacitor on EN (connected between EN and GND) fix the problem
@projectgus and all - thank you guys!
No more fiddling with EN / Boot buttons.
Life is much easier with this simple hack 😄
Krzysztof
hiya - got bit by this bug, and found a few things helped. capacitor helps but easier to change the second time.sleep(0.05) to time.sleep(0.5) e.g. https://github.com/adafruit/arduino-esp32/blob/master/tools/esptool.py#L279
that along with a formal 'reset' procedure to swat the ESP32 back into 'user mode'
https://github.com/adafruit/arduino-esp32/blob/master/tools/esptool.py#L296
has made developing code work great on win7 + CP210x... plz re-consider the wontfix? there's a lot of those DevKitC's in the world now :) i can test on mac/linux too
For information it is already fixed in actual branch even without the capacitor => thanks to an ESP32 silicon bug with reset by watchdog after 0.385s
I do not see why this modification will improve anything.
Do you have a sigrok capture to show the difference with this patch ?
Hi @bvernoux,
For information it is already fixed in actual branch even without the capacitor
I don't believe this is fixed in any branch at the moment, without adding either a capacitor or the time.sleep(0.5) modification that @ladyada mentions.
Hi @ladyada,
plz re-consider the wontfix? there's a lot of those DevKitC's in the world now :)
The problem I see is because the 0.5s delay happens to rely on a silicon bug, which is already fixed in the forthcoming chip revision. When the updated chips come out, the bug will bite everyone again (if the capacitor on EN hasn't been changed in the board design) and it will also take longer any time esptool.py fails to connect, due to the extra delays.
There are perhaps some other ways to work around this, either a command line option like --variant or --board to let people specify they have the gen1 silicon, or a long-standing suggestion (#27) to read back the "header" message that the chip prints in order to detect boot mode failures earlier.
yah ok - i didn't realize it was a silicon issue on the esp32 (thought it was cp210x) in which case... i agree y'all should keep as wontfix.
i will update the devkitc product page to tell people to add a capacitor if they're having upload probs on windows
yah ok - i didn't realize it was a silicon issue on the esp32 (thought it was cp210x) in which case... i agree y'all should keep as wontfix.
Just to be sure we're on the same page, the silicon issue on esp32 is what allows the extra time.sleep(0.5) delay to work as a workaround (the chip resets twice due to an unexpected WDT), rather than the cause of this bug. The cause of this bug is an interaction between the reset circuit timing and Windows driver behaviour (probably CP210x-specific but I haven't confirmed this.)
i will update the devkitc product page to tell people to add a capacitor if they're having upload probs on windows
That would be awesome, thanks very much. :)
ahh ok yes reading thru more carefully i understand the double-bug interaction! too funny - ok thanks!
just found my way to this issue. not before spending some time banging my head against the wall, though :)
the workaround seems fine, i just want to confirm: devkit hardware will be revised to increase the capacitor value, right? hopefully, in time for new chip revision which fixes the glitch.
Yes, newer Espressif DevKits will fix this issue via increased capacitance on the EN pin.
esptool v2.0 (current master, and the version in esp-idf) has a new --after esp32r0 option which will work around this issue for ESP32 original revision chips.
The option can also be selected under "make menuconfig" in esp-idf.
I've been investigating this and I think there's a more permanent solution that'd be effective and not too burdensome. Interestingly, the esp32r0 workaround doesn't seem to help at all for this yet it sounds like a similar issue if not the same one.
A long discussion follows with technical details I've gleaned, but the bottom line is that holding the chip in reset longer does the trick. The code change to esptool.py is a one-liner:
...
self._port.setDTR(False) # IO0=HIGH
self._port.setRTS(True) # EN=LOW, chip in reset
time.sleep(2.0) # Sleep longer
self._port.setDTR(True) # IO0=LOW
self._port.setRTS(False) # EN=HIGH, chip out of reset
time.sleep(0.5) # Sleep longer
...
This has the advantage of working on both current boards (and 2 seconds isn't long to wait relative to everything else). If it were a default it'd work smoothly with the Arduino-ESP32 as well - not sure how one is supposed to get --before esp32r0 to work there - is it a config option there (the reset-on-disconnect thing is of related interest but for other discussions).
Note: for the hard-reset bit, while I see the --before esp32r0 option, I don't see it in --after. Not sure if there is any problem with board reset (as long as DTS is in the right state) so maybe that was a typo?
The Core Board v2 and ESP-WROVER-KIT act differently upon serial disconnect, and downloads negotiation works slightly differently for each.
Previous discussions about this include:
espressif/esp-idf#59
#136
espressif/arduino-esp32#127 (comment)
http://esp32.com/viewtopic.php?f=13&t=334
Tested on Windows with the latest serial drivers via RealTerm:
Si Labs CP210x driver v6.7.4.261 (non-enumerating) for the Core Board v2
FTDI driver v2.12.24.0 (enumeration disabled) for the ESP-WROVER-KIT
Note: interestingly, clear/set can be positive logic (clear = 0/low/false, set = 1/high/true), or the inverse (active low). In esptool.py the active low mode is used - here, I used active high notation.
For the ESP-WROVER-KIT, to get to download mode, it's pretty simple:
- Start with RTS and DTR set
- Do the following in quick succession (no long pause between the steps):
- clear DTR (holds reset low)
- set DTR
- clear RTS (enters download mode)
- set RTS (can immediately do this)
- toggle DTR clear+set to reset in run mode
The Core Board v2 is different though - it needs about 2 seconds delay between clearing DTR and setting it:
- Start with RTS and DTR set
- Do the following in quick succession (except as noted):
- clear DTR (holds reset low)
- wait 2 seconds
- set DTR
- clear RTS (enters download mode)
- set RTS (can immediately do this)
- toggle DTR clear+set to reset in run mode
The boot up sequence differs slightly too - on the Core Board v2 that long pause causes a flash error.
ESP-WROVER-KIT
ets Jun 8 2016 00:22:57
rst:0x1 (POWERON_RESET),boot:0x16 (SPI_FAST_FLASH_BOOT)
ets Jun 8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x6 (DOWNLOAD_BOOT(UART0/UART1/SDIO_REI_FEO_V2))
waiting for download
Core Board v2
ets Jun 8 2016 00:22:57
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
Falling back to built-in command interpreter.
OK
>ets Jun 8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x3 (DOWNLOAD_BOOT(UART0/UART1/SDIO_REI_REO_V2))
waiting for download
Also peculiar: if DTR and RTS are both cleared, the EN switch on the ESP-WROVER-KIT won't work at all (this is not the case on the Core Board v2).
(As an aside, on both devices, if RTS is set, setting DTR (from cleared) resets the board. If DTR is clear, clearing RTS (from set) also resets the board.)
The only obvious difference between the two RTS/DTR circuits in the corresponding schematics is that the Core Board v2 specs 12kohm resistors between the CP2102 bridge and the bases of the S8050 transistors, while the ESP-WROVER-KIT specs 1kohm resistors between the FT2232HL bridge and the bases of the S8050 transistors. (The BOOT and EN switch circuits appear to be identical on both.)
Note: I have no shunts bridging the CTS/RTS headers on the WROVER board. It doesn't appear it would have a positive effect.
One other thing... I'm using non-enumerating drivers here. I strongly recommend people stay far away from the enumerating drivers (or disable enumeration for the ports using it), particularly in Windows, if they don't have a reason to be using it (typically emulating a mouse).
If the mouse filter gets spurious data (as is quite likely even from a factory fresh device during reset/boot-up) you could get a blue screen. Longer discussion here.
The default driver Windows installs for the CP2102 is enumerating - the one I used to test all this is the same one but non-enumerating. That alone could compound everything above as there is that extra layer of introspection for enumeration purposes (and they do act differently... it's annoying. My fix above should work in either case, though the risk of BSOD would remain with enumeration).
This is not windows specific. Same issue on macOS with cp210x drivers: espressif/esp-idf#305
Thanks to @MartyMacGyver and everyone else who helped diagnose this, we seem to have a reliable fix now which doesn't require any special command line arguments.
This fix should propagate to the ESP-IDF & Arduino esptool.py versions, soon.
Also, new development board hardware should no longer exhibit the underlying bug in the first place (provided 100nF or more of capacitance is present on the EN pin, with a 10K or higher pullup on that pin).
Please open an issue if this fix doesn't appear to fix auto-reset on your board (either reopen this issue, or if you have more specifics to provide then please open a new one instead.) Thanks.
Hi,
I've been able to fix this with 1uF capacitor from IO0 to GND. I have another on EN but the one that solves this bug on some silicon revisions are the 1uF capacitor on IO0.
Also I needed to put this on the platformio.ini configuration file to be able to use the serial monitor (in Arduino ide no fix is needed):
monitor_rts = 0
monitor_dtr = 0
FWIW, I have the NodeMCU ESP32S boards (https://www.amazon.ca/KeeYees-Development-Bluetooth-Microcontroller-ESP-WROOM-32/dp/B07QCP2451) that needed button pressing to program. A 0.1µF cap didn't change anything, a 2.2µF did the trick. ESPtool.py v2.8 under Linux.
[@projectgus not sure whether this is sufficient reason to open a new issue? I don't seem to be able to reopen this one]
Cross-link to similar problem for CP2102 chips and its solution:
#706 (comment)
Hi all,
Thanks for all the analysis and descriptions so far. I ran into the issue of auto-resetting an "ESP32S" module (based on ESP-WROOM-32 and CP2102) today on Linux (Debian 11.6) and looked into it a bit. I noticed that the CP2102 apparently shows some behavior regarding activating RTS and DTR lines that deviates from most other USB/serial converters I have used so far. It looks to me as if an activation command (ioctl) for one of RTS or DTR somehow resets the other line to the idle state. Some of the commands are apparently even ignored (that means do not change the pin states at all). Edit: I think, I was mistaken. It's actually some transistors on the dev board, which somehow connect the state of the RTS and DTR lines to each other. So it's not a CP2102 issue, but an issue of the transistors on the board. However, this makes the workaround described below a potential workaround for the transistor issue.
I do not know if it is a solution workaround that anybody else would like to use, but I manged to make it work for my case. The following patch to esptool.py (which is quite dirty because it's messing with pyserial internals) made the auto-reset work:
diff --git a/esptool.py b/esptool.py
index bea3555..44d3a21 100755
--- a/esptool.py
+++ b/esptool.py
@@ -459,7 +459,14 @@ class ESPLoader(object):
#
# DTR & RTS are active low signals,
# ie True = pin @ 0V, False = pin @ VCC.
- if mode != 'no_reset':
+ if mode == 'cp210x_transistor_linux_reset':
+ import fcntl
+ fcntl.ioctl(self._port.fd, serial.serialposix.TIOCMSET, serial.serialposix.TIOCM_RTS_str)
+ time.sleep(0.1)
+ fcntl.ioctl(self._port.fd, serial.serialposix.TIOCMSET, serial.serialposix.TIOCM_DTR_str)
+ time.sleep(0.1)
+ fcntl.ioctl(self._port.fd, serial.serialposix.TIOCMSET, serial.serialposix.TIOCM_zero_str)
+ elif mode != 'no_reset':
self._setDTR(False) # IO0=HIGH
self._setRTS(True) # EN=LOW, chip in reset
time.sleep(0.1)
@@ -2819,7 +2826,7 @@ def main(custom_commandline=None):
parser.add_argument(
'--before',
help='What to do before connecting to the chip',
- choices=['default_reset', 'no_reset', 'no_reset_no_sync'],
+ choices=['default_reset', 'no_reset', 'no_reset_no_sync', 'cp210x_transistor_linux_reset'],
default=os.environ.get('ESPTOOL_BEFORE', 'default_reset'))
parser.add_argument(
Flashing needs to be done with make flash CONFIG_ESPTOOLPY_BEFORE=cp210x_transistor_linux_reset in order to activate the special reset mode.
Some technical details as far as I understand them so far:
The pyserial module uses the TIOCMBIS and TIOCMBIC ioctl calls to modify one of the DTR and RTS bits at a time. Using those commands, I could not convince the CP2102 + transistors on the board to output signals with a timing that worked. Using the TIOCMSET ioctl call, it is possible to set RTS and DTR to a new state at the same time. The CP2102 + transistors still did not output the signals exactly as indicated in the command. (I do not know why. I don't even know if it should. the transistors...). However, I manged to find a sequence of commands that puts RTS to low (activate reset), and a bit later RTS to high and DTR to low (take back reset and indicate "flash mode" via GPIO0) within the same microsecond. Apparently, this is fast enough so that the ESP32 sees the GPIO0 at low (flash mode) when coming out of reset and auto-flashing works for me.
Hi @stefanschuermans,
a similar solution has been implemented just recently in 353cefc. Does this help in your case? You can try pulling the latest esptool or install it with pip install esptool==4.5.dev1.
Hi @radimkarnis. Thanks for this pointer. I have tried this version and it works for me as well. :-) That's great, so no I do not need a custom solution.
wie lad ich diese version herunter und wie kann die installiern bitte hilfe
之所以增加这一部分电路,是因为上位机编程,不管是python,还是C#,直接操作RTS与DTE是有延时的 ,一般情况直接操作不会有问题,但有时由于操作系统调度等会产生卡顿,延时不能精确控制,所以增加这部分电路用硬件去精确控制复位流程,还有就是减少烧写工具与esp的耦合,不会钳位IO引脚,经过我的测试,EN引脚的电容不宜过大,我开始用的10u的,太大了充电时间长,芯片复位的过程就长,等待时间就长,太小了起不到缓慢充电的作用,实际测试中由于我的引线过长,不焊接电容也能有电容的效果,所以如果不能自动进入复位流程就要增大电容,复位时间过长可以减小电容,以上