Persistent Boot Failure on ESP32-S3
Gspohu opened this issue · comments
Board
ESP32-S3-WROOM-1U-N4R2
Device Description
Plain module
Hardware Configuration
Irrelevant
Version
v3.0.1
IDE Name
PlatformIO
Operating System
Linux Ubuntu 22.04
Flash frequency
80
PSRAM enabled
yes
Upload speed
115200
Description
Hello,
I am experiencing a consistent boot failure issue with my ESP32-S3, regardless of the complexity or simplicity of the firmware I attempt to upload. This issue persists across both my pretty complex custom firmware and basic example code, such as the WiFi connection example. The behavior suggests a fundamental problem related to the boot process.
Issue Description
The device fails to boot properly. The error message that is consistently output is as follows:
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40377989
...
E (681) mmap: esp_mmu_unmap(595): munmap target pointer is outside external memory regions
assert failed: spi_flash_munmap flash_mmap.c:241 (0 && "invalid handle, or handle already unmapped")
Backtrace: 0x40377d1e:0x3fcf4610 ...
ELF file SHA256: fd4bb4dfa7adf1ca
My point of view is that it's linked to the model chip I have.
I'm currently stuck on this model of chip, but it's not the first time I've had to unravel memory-related boot problems with this model of chip ESP32-S3-WROOM-1U-N4R2.
Additional Context
- This issue began after I updated my ESP32-S3 with the latest release available on the GitHub repository, following a period of two months without updates since the device was functioning as needed without them.
- The problem persists even when go to the last commit on the master branch.
This seems to be a regression or a newly introduced bug associated with the latest updates. Any guidance or suggestions on how to mitigate this issue would be greatly appreciated. Thank you for your attention to this matter.
Sketch
NoneDebug Message
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40377989
SPIWP:0xee
mode
, clock div:1
load:0x3fce3818,len:0x508
load:0x403c9700,len:0x4
load:0x403c9704,len:0xad0
load:0x403cc700,len:0x29e4
entry 0x403c9880
E (681) mmap: esp_mmu_unmap(595): munmap target pointer is outside external memory regions
assert failed: spi_flash_munmap flash_mmap.c:241 (0 && "invalid handle, or handle already unmapped")
Backtrace: 0x40377d1e:0x3fcf4610 0x4037d915:0x3fcf4630 0x403838a5:0x3fcf4650 0x4201d19b:0x3fcf4780 0x4201cbd1:0x3fcf47a0 0x4201ccd5:0x3fcf47c0 0x4201d06f:0x3fcf47f0 0x42014784:0x3fcf4850 0x420167a2:0x3fcf4890 0x420c4ac9:0x3fcf48c0 0x4038064a:0x3fcf4900
ELF file SHA256: fd4bb4dfa7adf1ca
Other Steps to Reproduce
No response
I have checked existing issues, online documentation and the Troubleshooting Guide
- I confirm I have checked existing issues, online documentation and Troubleshooting guide.
Yes, I looked carefully before posting the issue and it seems that I'm the first with this problem.
I can't go back to version 2, I've already migrated my firmware to version 3 and it's not possible to build it.
As I'm an idiot, when I updated I didn't look to see which commit I was on...
I'm currently testing them one by one to see when the bug appeared. If I find it, I'll post the info here.
If it can help here my plaformio.ini :
[env:esp32-s3-N4]
framework = arduino
board = adafruit_feather_esp32s3
monitor_speed = 9600
platform = https://github.com/platformio/platform-espressif32.git
platform_packages =
framework-arduinoespressif32@https://github.com/espressif/arduino-esp32.git
framework-arduinoespressif32-libs@https://github.com/espressif/esp32-arduino-libs.git#idf-release/v5.1
lib_deps =
bblanchon/ArduinoJson@^6.19.4
WiFi
build_flags =
-DCONFIG_LWIP_IPV4_NAPT=1
-DIP_NAPT=CONFIG_LWIP_IPV4_NAPT
-DCONFIG_LWIP_IP_FORWARD=1
-DCONFIG_LWIP_L2_TO_L3_COPY=1
-DCONFIG_IDF_TARGET_ESP32S3=1
-DARDUINO_USB_MODE=1
-DARDUINO_USB_CDC_ON_BOOT=1
-DCONFIG_LWIP_IPV6=1
-DCONFIG_LWIP_IPV6_DHCP6=1
-DCONFIG_LWIP_IPV6_AUTOCONFIG=1
remove the not working settings in build_flags. This are compile time settings only working with IDF. The are useless with Arduino. Pin the used Arduino libs with commit SHA
[env:esp32-s3-N4]
framework = arduino
board = adafruit_feather_esp32s3
monitor_speed = 9600
platform = https://github.com/platformio/platform-espressif32.git#40cf417aa2e2d32b32c5e9540c458e8869c5ec58
platform_packages =
framework-arduinoespressif32@https://github.com/espressif/arduino-esp32.git#cbf1e94dd5f9ac8bdabaae9603611a3b81d8fbe4
framework-arduinoespressif32-libs@https://github.com/espressif/esp32-arduino-libs.git#a51b0d7fd9bffef217f98d1bc0bedca327bc83ea
lib_deps =
bblanchon/ArduinoJson@^6.19.4
build_flags =
-DARDUINO_USB_MODE=1
-DARDUINO_USB_CDC_ON_BOOT=1
I personally wouldn't use the Adafruit boards.json at all since it uses the imho flash wasting tinyuf2
OK I'm going to test the build flag, but it's not my choice to use Adafruit board json, for this specific chip I don't know what to use as a board file. If you have one to recommend to me, I'm open !
Edit :
I've removed the non-arduino build flag and that doesn't change the problem
copy this one in folder boards in root of your project and give it a name like S3-WROOM-1U-N4R2.json to use in Platformio: board = S3-WROOM-1U-N4R2
{
"build": {
"arduino": {
"memory_type": "qio_qspi"
},
"core": "esp32",
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"mcu": "esp32s3",
"variant": "esp32s3"
},
"connectivity": [
"wifi",
"bluetooth"
],
"debug": {
"openocd_target": "esp32s3.cfg"
},
"frameworks": [
"arduino",
"espidf"
],
"name": "WROOM-1-N4R2",
"upload": {
"flash_size": "4MB",
"maximum_ram_size": 327680,
"maximum_size": 4194304,
"require_upload_port": true,
"speed": 460800
},
"url": "https://www.espressif.com",
"vendor": "espressif"
}
If it still bootloops one of your used libraries is to blame. Probably the breaking change PR #9453 introduces problems in many extern libraries
Can you disable anything related to PSRAM?
Yesterday a user of my software also reported some issues (unclear yet what actually is the issue), and he had also one of the newer S3 chips with 4M flash and 2M PSRAM. (his board uses ESP32-S3FH4R2 )
And since the S3 does have some issues with adressing flash/PSRAM, maybe disabling PSRAM is worth a try to narrow down the issue.
H4R2 is High Temp flash and psram, N4R2 is Normal Temp (85C). @TD-er can you get more info from your user?
@me-no-dev
I will ask him to do some more tests to figure out what actually went wrong.
As far as I know is that he reported he is unable to flash this board using the web flasher.
But I figured that if he also might have some bootlooping board after an initial flash, that it would easily look like "unable to flash"
It is this board:
I've tested it with @Jason2866 new board file (thanks for that), but the bug is exactly the same.
BUT I disabled the PSRAM by removing the flag in the board file and the problem disappeared. So it's the PSRAM management that's at fault here.
It's a bit off the subject for this issue (if I find the time I'll do an issue on it), but on master the WiFi isn't working at all, and it's impossible to enable WPS. I think it's because there is no more ram
How was the PSRAM configured?
I think it was configured to use OPI wiring.
Can you compare your board JSON with what I'm using for 4M S3 boards:
{
"build": {
"arduino":{
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_qspi"
},
"core": "esp32",
"extra_flags": "-DARDUINO_TASMOTA -DBOARD_HAS_PSRAM -DARDUINO_USB_MODE=1 -DUSE_USB_CDC_CONSOLE -DESP32_4M -DESP32S3 -DARDUINO_USB_CDC_ON_BOOT=1",
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"hwids": [
[
"0x303A",
"0x1001"
]
],
"mcu": "esp32s3",
"variant": "esp32s3",
"partitions": "boards/partitions/esp32_partition_app1810k_spiffs316k.csv"
},
"connectivity": [
"wifi",
"bluetooth",
"ethernet"
],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": [
"esp-builtin"
],
"openocd_target": "esp32s3.cfg"
},
"frameworks": [
"espidf",
"arduino"
],
"name": "Espressif Generic ESP32-S3 USB CDC >= 4M Flash QSPI PSRAM, ESPEasy 1810k Code/OTA 316k FS",
"upload": {
"flash_size": "4MB",
"maximum_ram_size": 327680,
"maximum_size": 4194304,
"require_upload_port": true,
"before_reset": "usb_reset",
"speed": 460800
},
"url": "https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/",
"vendor": "Espressif"
}You may have some options different, like partitions and maybe some specific flags.
You can see mine here : #9874 (comment)
I just remove "-DBOARD_HAS_PSRAM",
Then I honestly have no idea why it would not work with PSRAM enabled.
I took the time to make a more methodical study of the problem, to understand when it first appeared.
So I went back to the version where I was before the problem appeared.
Here's a table summarising my research/tests:
The error message I talk about is :
[ 5584][E][NetworkManager.cpp:130] hostByName(): DNS Failed for 'xxx.eu' with error '-54'
[ 5594][E][ssl_client.cpp:116] start_ssl_client(): connect on fd 48, errno: 118, "Host is unreachable"
[ 5604][E][NetworkClientSecure.cpp:159] connect(): start_ssl_client: connect failed: -1
[ 5613][E][NetworkClient.cpp:315] setSocketOption(): fail on 0, errno: 9, "Bad file number"
[ 5621][E][NetworkClient.cpp:315] setSocketOption(): fail on 0, errno: 9, "Bad file number"
E (21109) wifi:sta is connecting, return error
My board file is exactly this one : #9874 (comment)
In conclusion, the start problems linked to PSRAM really start between commit e382746 and commit cbf1e94 dated 13/06/2024.
If this speaks to anyone, I'm listening, because I don't know this code base enough to draw anything from my study. Lastly, is it possible that this is linked to precompiled libs?
is it possible that this is linked to precompiled libs?
It is, because we did not change anything that can influence the result. You say you can not debug, how come? What do you get in Serial?
Thanks for the feedback, @me-no-dev.
You mentioned that the issues might be linked to precompiled libraries, as there have been no changes that could directly affect this outcome. However, I'm facing a lack of serial output post-crash, which complicates pinpointing the exact cause.
Here the only message I get from the serial :
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40377989
SPIWP:0xee
mode
, clock div:1
load:0x3fce3818,len:0x508
load:0x403c9700,len:0x4
load:0x403c9704,len:0xad0
load:0x403cc700,len:0x29e4
entry 0x403c9880
E (681) mmap: esp_mmu_unmap(595): munmap target pointer is outside external memory regions
assert failed: spi_flash_munmap flash_mmap.c:241 (0 && "invalid handle, or handle already unmapped")
Backtrace: 0x40377d1e:0x3fcf4610 0x4037d915:0x3fcf4630 0x403838a5:0x3fcf4650 0x4201d19b:0x3fcf4780 0x4201cbd1:0x3fcf47a0 0x4201ccd5:0x3fcf47c0 0x4201d06f:0x3fcf47f0 0x42014784:0x3fcf4850 0x420167a2:0x3fcf4890 0x420c4ac9:0x3fcf48c0 0x4038064a:0x3fcf4900
ELF file SHA256: fd4bb4dfa7adf1ca
And it's restart loop like this.
Here’s a quick rundown of what I’m observing:
- The system fails to boot after firmware updates
- Errors seem related to PSRAM management, as indicated by error messages and improved behaviour when PSRAM is disabled
Could you clarify if there have been any recent changes in the ESP-IDF that might affect memory or PSRAM handling specifically? Or where I can search for that ? Maybe I need to post my issue somewhere else ?
Also, are there recommended tools or methods for diagnosing this issue more effectively in the absence of clear serial debug outputs?
Could you clarify if there have been any recent changes in the ESP-IDF that might affect memory or PSRAM handling specifically?
I can not. We can not follow the changes in ESP-IDF. Big project, many people and changes.
Maybe I need to post my issue somewhere else ?
ESP-IDF's issue tracker is a good place to post such issue with all details that you have figured out thus far.
2.0.17 is on IDF v4.4.7, 3.0.1 is on IDF v5.1.4
Ok, I will try to post my issue over there.