[BUG] Freeze issues (Marlin M43, M122, on ESP3D 3.0)
rondlh opened this issue · comments
Describe the bug
ESP3D 3.0 seems to have an issue with TABS (ASCII 9), when I run a M122 (Marlin get TMC info, which returns some Tabs) then ESP3D freezes.
I also see the same thing when using M43 (pin debug info, which responds with a lot of data (8-9KB). It sometimes works, and sometimes freezes ESP3D. Also the Marlin startup info can cause the same issue (also a lot of lines), when I restart Marlin after ESP3D is connected.
To Reproduce
- Connect to Marlin machine and run M122, M43, or restart Marlin when ESP3D is connected.
- M122 cause ESP3D to freeze 2 lines are shown on the terminal, M43 locks up within a few tries, some data is usually received.
Expected behavior
No freezes
ESP3D Firmware:
- ESP3D 3.0, latest (Feb 1st 2024) with latest Marlin Webui.
- Wifi mode: STA
- Flash method: PlatformIO, OTA flash.
Current Marlin build (Jan 2024).
NOTE: ESP3D 2.1.x does not have this issue.
Board used:
- MCU: ESP32-S 2AHMR
- Name: NodeMCU V1.3 Dev module
- Flash size: 4MBytes
Additional context
Connection layout: Marin <--250K Baud--> BTT TFT35V3 <--1M Baud--> ESP3D
Thank your for submiting, please be sure you followed template or your issue may be dismissed. if you deleted the template it is here
can you provide an output from Marlin so I can reproduce
Do you have same issue if using telnet ?
Do you have same issue using TT TFT35V3 <--250K Bauds --> ESP3D ?
can you provide an output from Marlin so I can reproduce Do you have same issue if using telnet ?
The data seems to be processed very slowly, in blocks of about 1.2KB with long (1-2 seconds) pauses in between.
M43_Putty.txt
In this Telnet Putty capture the first M43 worked, the 2nd M43 caused a freeze.
Note: In Putty I have enabled "Implicit CR in every LF".
Do you have same issue using TT TFT35V3 <--250K Bauds --> ESP3D ?
Yes, I still have the same problem at 250K Baud.
With telnet M43 still causes a freeze.
For M122 I get 4 line instead of 2 on the browser, but it still freezes.
M122 OUTPUT:
M122
X Y Y2 Z Z2 Z3 Z4 E
Address 0 0 0 0 0
Enabled false false false false false false false false
Set current 1000 901 902 801 802 803
what version of Marlin are you using ? release or bugfix ?
can you share your configuration.h and configuration_adv.h please?
what are the exact version in webui for web ui?
do you have same issue if you conect esp board directly to printer board?
Do you have same issue with M115 and M503?
what version of Marlin are you using ? release or bugfix ? can you share your configuration.h and configuration_adv.h please? what are the exact version in webui for web ui? do you have same issue if you conect esp board directly to printer board? Do you have same issue with M115 and M503?
-
I use a recent Marlin bugfix. See attached configuration files. My configuration is a bit complex, I manage 4 printers with it. I'm testing with the selected IRON_TRONXY configuration. Note that there is no issue on the Marlin side, no freezes or any issues.
Configuration.zip -
WebUI version 3.0.0-a51.M2.1 (the latest version as far as I know)
-
UPDATE: I connected the ESP32 to the MKS Monster8 V1 motherboard directly, and found the same issues. M122, M43, M503 all cause freezes.
-
M115 generates about 1090 bytes, all data seems to arrive in 1 batch, M115 also causes freezes, then no data arrives at all after the M115
-
M503 generates about 2726 bytes, and also causes freezes. The data seems to arrive in 3 parts with long delays in between, even for the first data.
Signal strength is reported to be 94%, 11n, channel 12.
thank you for the feedback , connect the esp32 on board instead of tft would be also a good check to see if issue is due to tft bridging data or serial of esp3d getting corrupted
thank you for the feedback , connect the esp32 on board instead of tft would be also a good check to see if issue is due to tft bridging data or serial of esp3d getting corrupted
I tried it, but got the same result, ESP3D freezes. The issues with M122 should be easy to reproduce, right?
I just did some tests to see were the issue is.
1. I send a long ESP command, that mirrors the data back, like:
[ESP111]<10KBytes_DATA_string>
This works well and smoothly, the data is received very fast.
Note: The long string has no '\n'
2. I let a ESP command generate a lot of lines, 200 lines of 60 bytes ending on '\n'
This also works very well and is very fast.
3. I increased the Serial RX buffer to 10KBytes and let the ESP wait until the whole M43 response is received before reading the RX buffer. (wait to read data until "available bytes" doesn't change for 25ms).
The data is received very slowly/choppy and the ESP sometimes freezes.
UPDATE: I think I know why M122 doesn't work:
In serial_service.cpp there is this line that analyses the received serial data:
else if (isPrintable(char(sbuf[i]))) {
A TAB (ASCII Char 9) is not "printable".
It works correctly if I replace this line with:
else if ((sbuf[i] == 9) || (sbuf[i] > 31) && (sbuf[i] < 127)) {
wow good catch
I have updated the code I hope it cover all needs : e471be5
Thank you for the clean analyzis
That was quick, good job.
This solves the M122 issue, but do we actually understand why this causes a freeze?
For the TAB character this code would be executed:
} else { // it is not printable char
// clean buffer first
if (_buffer_size > 0) {
flushbuffer();
}
// process char
_buffer[_buffer_size] = sbuf[i];
_buffer_size++;
flushbuffer();
}
This flushes the buffer (if data is available) before copying the TAB to the now empty buffer, then the buffer (with only the TAB) is flushed again.
In "flushbuffer" a message is created with the buffer which is processed as a ESP3D command. Then the buffer is reset.
In the "process" function the data is dispatched, so the part before the TAB and the TAB itself as a single character.
Why would this cause the ESP to freeze?
To be honest I am not sure neither - it should not freeze but skip the buffer - I will review this soon more deeply- Initially was to detect corrupted data but I did not do this on ESP3D-TFT and it is ok
I suspect some dispcrepency between 2 buffers but did not look into yet
I will use your data to reproduce and simulate Marlin output because I do not have TMC system in hand
The freeze issue does not seem to be related to the M43 output, but seems to be related to the rapid fire of serial messages.
I made a quick test function, that can be use as a replacement for Marlin's M119.
M119 without parameters works normally.
Parameter D defines the delay between the lines (use 0-35), default is 35ms
Parameter L defines the number of lines to generate, default is 100 lines.
The Marlin watchdog timeout needs to be considered, so disable it or set WATCHDOG_DURATION_8S to get more playroom.
Example: "M119DL" (generates 100 lines with 35ms delay between the lines)
My current conclusions:
- Long messages (10KB) without line feeds are processed without a problem. (adjust ESP3D RX buffer!)
- Quick messages with an interval of about 30ms can be processed without any issues.
- Quick messages faster than 30ms become unreliable, processing slows done and ESP3D 3.0 freezes occur.
void GcodeSuite::M119() {
if (parser.seen('D')) // D=DELAY(35), L=LINES(100)
{
uint32_t d = parser.has_value() ? parser.value_int() : 35; // Default 35ms delay
if (parser.seen('L')) { // L=LINES(100)
uint32_t l = parser.has_value() ? parser.value_int() : 100; // Default 100 lines
#ifdef USE_WATCHDOG
#ifndef WATCHDOG_DURATION_8S
if ((d * l) > 3500) SERIAL_ECHOLNPGM("WARNING: POTENTIAL MARLIN WATCHDOG TIMEOUT, DEFINE WATCHDOG_DURATION_8S");
#else
if ((d * l) > 7000) SERIAL_ECHOLNPGM("WARNING: POTENTIAL MARLIN WATCHDOG TIMEOUT (8000ms)");
#endif
#endif
SERIAL_ECHOLNPGM("DELAY:", d, "ms LINES:", l);
for (uint32_t i = 0; i < l; i++)
{
SERIAL_ECHOLNPGM("AAAAAAAAAA AAAAAAAAAA AAAAAAAAAA AAAAAAAAAA AAAAAAAAAA AAAAAAAAAA AAAAAAAAAA AAAAAAAAAA ", i);
if (d) delay(d);
}
SERIAL_ECHOLNPGM("DELAY:", d, "ms LINES:", l);
}
}
else
endstops.report_states();
}
I'm not sure yet, but it seems the issue is the 100ms timeout below:
size_t ESP3DSerialService::writeBytes(const uint8_t *buffer, size_t size) {
if (!_started) {
return 0;
}
if ((uint)Serials[_serialIndex]->availableForWrite() >= size) {
return Serials[_serialIndex]->write(buffer, size);
} else {
size_t sizetosend = size;
size_t sizesent = 0;
uint8_t *buffertmp = (uint8_t *)buffer;
uint32_t starttime = millis();
// loop until all is sent or timeout
while (sizetosend > 0 && ((millis() - starttime) < 100)) {
size_t available = Serials[_serialIndex]->availableForWrite();
if (available > 0) {
// in case less is sent
available = Serials[_serialIndex]->write(
&buffertmp[sizesent],
(available >= sizetosend) ? sizetosend : available);
sizetosend -= available;
sizesent += available;
starttime = millis();
} else {
ESP3DHal::wait(5);
}
}
return sizesent;
}
}
did you changed the value to check ?
Also what is the setting in ESP3D you have for this :
/* Add serial task
* ESP32 need to add a task to handle serial communication
*/
// #define SERIAL_INDEPENDANT_TASK
Is it commented or not ?
did you changed the value to check ?
I changed the timeout value to 1000, then the problem seems to go away.
Also what is the setting in ESP3D you have for this :
/* Add serial task * ESP32 need to add a task to handle serial communication */ // #define SERIAL_INDEPENDANT_TASK
I tried both, commented out and uncommented. I cannot see any difference in respect to the freeze problem.
Should it be uncommented as I'm using an ESP32?
before using dedicated task was necessary to not loose incoming serial data, now it should not be necessary with latest arduino release, but I kept it for safety in case of weird behavior would appear
So 1000ms seems solving issue ? that is good news
before using dedicated task was necessary to not loose incoming serial data, now it should not be necessary with latest arduino release, but I kept it for safety in case of weird behavior would appear
OK, thanks.
So 1000ms seems solving issue ? that is good news
Yes, but I need to look into it closer to find out why the ESP freezes, perhaps there is a buffer overrun or something.
What I see is that the ESP starts lagging behind and larger and larger blocks (4KB) of serial input data are read at a time.
esp3d split messge by \r and \n, I do not understand how you can need a 4KB data at once
esp3d split messge by \r and \n, I do not understand how you can need a 4KB data at once
In "void ESP3DSerialService::process()", where the serial data is actually read, large blocks of data become available when the ESP lags behind. This data contains many lines.
check readBytes function, it is from serial buffer
check readBytes function, it is from serial buffer
Check what specifically?
sorry read on the phone because outside I wrongly read you asked the question Where?
OK, still testing here, not sure if my conclusions are accurate, it seems the current Wifi performance also plays a role.
I just noticed that if I close the browser window, then the telnet session works very fast without problems. So the issue seems to be on the WebSocket side. I guess that is what you wanted me to test when you asked me if I the problem also occurs in Telnet.
Tomorrow I will start to setup an Octopus which is the similar as Chip as MKS Monster, I will put some TMC2209 drivers - and install / configure Marlin to reproduce the issue - I have an esp32 module for the BTT ESP socket =>TBC
I was busy on others topics until now sorry,
wifi performance definitly have impact on speed transfer, it may slow down the transfer, but buffers should compensate, I have also a python simulator for Marlin to stream gcode : https://github.com/luc-github/ESP3D-TFT/tree/main/tools/fw_simulator if we need to fine tune the different serial packets output size
If you want to give a try to this : https://github.com/luc-github/ESP3D/tree/dabc95bc8c9fccd00d3dcb93455bae93e324ed66
it is ESP3D 3.0 before I did a big refactoring (last month) which may also be a root cause as one of the refactoring topic is the rewrite of internal messaging
If you want to give a try to this : https://github.com/luc-github/ESP3D/tree/dabc95bc8c9fccd00d3dcb93455bae93e324ed66
it is ESP3D 3.0 before I did a big refactoring (last month) which may also be a root cause as one of the refactoring topic is the rewrite of internal messaging
You mean I can try this branch to see if I have the freeze/slow down issue?
yes
Test done with Marlin bugfix (my fork) + ESP3D 200 (before refactoring) because it was the version already present on ESP32 module => No issue , I did several M122 / M43 commands when polling commands is enabled and so far it is ok.
Indeed the reporting is pretty huge
Now I will test with latest ESP3D 3.0 with client refactoring
I did several times but no issue with latest ESP3D neither
even removing antenna so got 66% signal
And I did not changed the 100ms timeout
Can you share how you connect the ESP32 board to the MKS Monster ?
Edit: I have also used your settings : #define RX_BUFFER_MONITOR / #define EMERGENCY_PARSER / #define ADVANCED_OK
and no problem M122/M43 respond and no freeze
but I was not able to use :
/* This feature is an EXPERIMENTAL feature so it shall not be used on production
* machines. Enabling this will allow the reception of serial data on the serial
* ports to be handled by the onboard DMA controller. This allows for more stable
* and reliable serial data reception at higher speeds. Currently only
* STM32F0xx, STM32F1xx, STM32F2xx, STM32F4xx, STM32F7xx are supported.
* This feature does not affect emulated USB serial ports.
*/
#define SERIAL_DMA // IRON, STM32F4xx DMA READING SUPPORTbecause the port used by ESP seems not like it and does not compile, you give also a try disabling it
Thanks for the great efforts.
It's seems the timeout I mentioned before is not the issue. I guess it was just a coincidence that it started working when I adjusted the timeout. M43 still gives me issues, usually after a few runs, but now I'm testing with the M119 code I posed above. M119D0L will cause a freeze within a few tries.
When I connected the ESP directly to the USART I use the dedicated UART PORT, see picture (red box).
SERIAL_DMA is new in Marlin, I worked with the team to add it in Marlin after adding it to the BTT TFT Touchscreen software. Current support is only for the STM32Fx series, so it depends on your hardware. It improves serial data reliability significantly.
I will do some tests without it, and also try stressing ESP3D 2.1, and try your branch mentioned above.
I'm currently trying to find out where the code actually freezes/slows down by using the indicator LED. I will report back...
Note that I increased the serial RX buffer size to 10KB, otherwise you will probably not be able to capture the whole output of M43.
Also, without your recent "isPrintable" fix, M122 would always freeze within a few lines.
I found the problematic code:
In websocket_server.cpp, function: bool WebSocket_Server::isConnected()
This line:
return _websocket_server->connectedClients(true) > 0;
If I comment it out then M122 suddenly works fine. Calling this function too fast seems to be causing me issues. I will try to put a time guard around it to limit the call frequency.
Indeed there is some overflow and data lost:
if I do M43 on serial terminal to BTT Board I get 169 lines for 15140 chars
if I do M43 on webui terminal I get 22 lines for 3808 chars
if I enable esp32 independant serial task and I do M43 on webui terminal I get 80 lines for 7124 chars
if I change #define ESP3D_SERIAL_BUFFER_SIZE 1024 to 10000 and I do M43 on webui terminal I get 60 lines for 5197 chars
if I keep 10000 for ESP3D_SERIAL_BUFFER_SIZE and disable esp32 independant serial task and I do M43 on webui terminal I get 60 lines for 5075 chars
if I change serial buffer to 4096 and I do M43 on webui terminal I get 59 lines for 5199 chars
So as you found the websocket is also a bottleneck because its TX buffer is 1400K each 500ms
I also realize there several way currently to overflow the WebSocket buffer - I need to dig in
That said I never get any freeze which as you get several , is a caracteristic of buffer / memory overflow
Ok to confirm your finding and correct some
Serial using independant task do not lose data so far using 1024B buffer as by default, I confirmed using putty with webui closed and get all data:
the difference is due to messages are trimmed so trailling spaces are removed
So main issue is Websocket server overloaded and blocking others tasks, serial lose of data is solved re-enabling the serial as independant task
so may need to isolate websocket server in separate task also and review the TX buffer usage that may also overloaded
Indeed there is some overflow and data lost: if I do M43 on serial terminal to BTT Board I get 169 lines for 15140 chars if I do M43 on webui terminal I get 22 lines for 3808 chars if I enable esp32 independant serial task and I do M43 on webui terminal I get 80 lines for 7124 chars if I change #define ESP3D_SERIAL_BUFFER_SIZE 1024 to 10000 and I do M43 on webui terminal I get 60 lines for 5197 chars if I keep 10000 for ESP3D_SERIAL_BUFFER_SIZE and disable esp32 independant serial task and I do M43 on webui terminal I get 60 lines for 5075 chars if I change serial buffer to 4096 and I do M43 on webui terminal I get 59 lines for 5199 chars
So as you found the websocket is also a bottleneck because its TX buffer is 1400K each 500ms
I also realize there several way currently to overflow the WebSocket buffer - I need to dig in
That said I never get any freeze which as you get several , is a caracteristic of buffer / memory overflow
My M43 output is below 9KB, so if I set the RX buffer to 10KB then I can get the full report without data loss. You seem to need an even higher number (16KB). UPDATE: I can try with independent task enabled. I never enabled that to solve the data loss issue, only to check the freeze issue. I just started testing ESP3D 3.0 a few days ago. I like it, better usability than V2.1, but I also found some behavior I don't like. Liked "[ESP111]ABC" doesn't mirror the "ABC" back anymore. Not sure if it's a bug or feature.
Could it by I use a different SDK than you? PlatformIO reports this:
Resolving esp32dev dependencies...
Platform espressif32 @ 6.2.0 (required: espressif32 @ 6.2.0)
├── framework-arduinoespressif32 @ 3.20008.0 (required: platformio/framework-arduinoespressif32 @ ~3.20008.0)
├── tool-esptoolpy @ 1.40501.0 (required: platformio/tool-esptoolpy @ ~1.40501.0)
├── tool-mkfatfs @ 2.0.1 (required: platformio/tool-mkfatfs @ ~2.0.0)
├── tool-mklittlefs @ 1.203.210628 (required: platformio/tool-mklittlefs @ ~1.203.0)
├── tool-mkspiffs @ 2.230.0 (required: platformio/tool-mkspiffs @ ~2.230.0)
├── tool-openocd-esp32 @ 2.1100.20220706 (required: platformio/tool-openocd-esp32 @ ~2.1100.0)
├── toolchain-riscv32-esp @ 8.4.0+2021r2-patch5 (required: espressif/toolchain-riscv32-esp @ 8.4.0+2021r2-patch5)
└── toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5 (required: espressif/toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5)
I'm testing the code below, which limits the call frequency to 20 calls a second. This seems to work, there are many call of "isConnected" in the code, so perhaps another abstraction layer will be better. And I can have a look a layer lower to see if I can find some issues.
static uint32_t lastCheckTime = 0; // IRON, LIMIT CALL FREQUENCY TO PREVENT LOCKUP/FREEZE ON ESP32
static bool status = false;
bool WebSocket_Server::isConnected(){
if (_websocket_server && (millis() - lastCheckTime > 50)) {
status = _websocket_server->connectedClients(true) > 0;
lastCheckTime = millis();
//return _websocket_server->connectedClients(true) > 0;
}
return status;
}
Liked "[ESP111]ABC" doesn't mirror the "ABC" back anymore. Not sure if it's a bug or feature.
in 3.0 syntax changed https://esp3d.io/esp3d/v3.x/documentation/commands/esp111/index.html
Could it by I use a different SDK than you? PlatformIO reports this:
I also use platformio and have same versions
I'm testing the code below
I will review the complete websocket module code tomorrow (it is almost 7pm here and my wife is calling for diner) - limit the number of call of connectedClients is good finding, I reall appreciate your reporting and finding, as on my side I only have the data lose issue not the freezze 😓 so you really help a lot
Thank you again
in 3.0 syntax changed https://esp3d.io/esp3d/v3.x/documentation/commands/esp111/index.html
So it it still possible to output anything in front of the IP address, or only "M117"?
I'm working on integrating ESP3D into the BTT TFT Touchscreen firmware.
https://github.com/bigtreetech/BIGTREETECH-TouchScreenFirmware
It seems we are in the same timezone (I'm in Guangzhou). Anyway, happy to help!
no M117 is only added if OUTPUT=PRINTER is added - no other header is currently used
Do you need to add another thing ?
BTT Touch already support ESP3D : https://github.com/bigtreetech/BIGTREETECH-TouchScreenFirmware/pull/648/files#diff-b37cdfe5ce512aba7968f156fa34849966931ea628959413ffa45d3c69a3c09e
Or it may be broken again
What feature do you add ?
I'm in Guangzhou
I went in Shenzen several years ago for work and got oppotunity to visit several factories in guanzhou, big area ^_^
The TFT requests the IP address from the ESP3D V2.1 by sending "[ESP111]M117 IP:", which results in "M117 IP:192.168.1.xxx." This info is displayed on the TFT, then the TFT requests the time, which is used to timestamp the notifications and also can give an ETA on the ongoing print.
Of course I can get the IP address by doing a RegEX on the TFT serial input, but I do not want to do that for all data received, so just checking for "IP:" was very easy.
Is it possible for ESP3D 3.0 configure it to always respond with M117 ?
The lines below in ESP111 (V3) look a bit odd to me, the first line seems useless.
tmpstr = get_clean_param(msg, cmd_params_pos);
tmpstr = get_param(msg, cmd_params_pos, "OUTPUT=");
BTW: Is there any reason why the ESP doesn't support 1M baud serial bit rate. I'm using 1M baud for a long time already, working flawlessly.
I worked in Shenzhen Pingshan area for some time, but I'm already stationed in Guangzhou for a long time now, I'm Dutch.
I just checked the documentation for connectedClients:
int WebSocketsServerCore::connectedClients(bool ping = false)
count the connected clients (optional ping them)
Parameters:
ping – bool ping the connected clients
So the parameter defines if the clients should be pinged.
I can imagine that pinging a client 100 times in a very short time can cause some issues for the network/memory use/stack. Setting the parameter to "false" also solves my issue, that's perhaps the simplest solution.
return _websocket_server->connectedClients(true) > 0;
changed to
return _websocket_server->connectedClients(false) > 0;
If you need to ping the clients, then some limits should be introduced.
I checked the code to see if similar issues exist (telnet/BT/notifications/serial2socket), but didn't find any issue there.
BTW: One line in the Charts Panel says "Show Redondant", that should be "Show Redundant". The Charts panel gives me an additional empty chart below the "Entruder" and "Bed" chart.
The TFT requests the IP address from the ESP3D V2.1 by sending "[ESP111]M117 IP:", which results in "M117 IP:192.168.1.xxx." This info is displayed on the TFT, then the TFT requests the time, which is used to timestamp the notifications and also can give an ETA on the ongoing print.
Of course I can get the IP address by doing a RegEX on the TFT serial input, but I do not want to do that for all data received, so just checking for "IP:" was very easy.
I have added OUTPUT=PRINTER to solve several issue the M117 may no fit all targets fw, the space is handled differently in ESP3D 3.0 so [ESP111]M117\ the space will be trimmed and M117 won't be recognize on some FW, in your case it is different you added IP: but that was not the original purpose
I guess adding HEADER= as additionnal parameter is not big issue but I have hard time to understand the relationship between IP:192.168.2.5 and timestamp for printing ^_^
Is it possible for ESP3D 3.0 configure it to always respond with M117 ?
You mean [ESP111] ? that is weird if your are querying from telnet / websocket / webui, I prefere let user to do what it need and avoid most automatic choice that lead to complex code to handle all exeptions
The lines below in ESP111 (V3) look a bit odd to me, the first line seems useless.
tmpstr = get_clean_param(msg, cmd_params_pos);
tmpstr = get_param(msg, cmd_params_pos, "OUTPUT=");
You are right that was part of refactoring command I missed that ones, thank you for pointing out
only
tmpstr = get_param(msg, cmd_params_pos, "OUTPUT=");
is necessary as it a replacement of previous one (fixed)
BTW: Is there any reason why the ESP doesn't support 1M baud serial bit rate. I'm using 1M baud for a long time already, working flawlessly.
Hmm I just missed to add it when it become a standard in arduino, when I started coding ESP the terminal did not go more than 250000, then there was update for 500000, then esp32 was able to use 921600 for upload and MKS protocol was using 1958400 so I added them, I just missed to add 1000000 and 2000000 (fixed)
If you need to ping the clients, then some limits should be introduced.
I used the ping to detect early disconnected clients , because if not it was slowing down the communications - may be a ping every sec is ok using your function ?
Also I think I remember now why I do not flush every message - it is because was not working well, I though I was overloading websocket, so I flushed only every 500ms instead but now you mention the connected(true)problem it could be indeed the problem
BTW: One line in the Charts Panel says "Show Redondant", that should be "Show Redundant".
Yes my french sometimes cover my bad english thank you for pointing out (fixed)
The Charts panel gives me an additional empty chart below the "Entruder" and "Bed" chart.
That the sensors chart - you can hide it in settings if you do not have any
I confirm :
With 1024 Serial buffer - not changing any others buffer (websockets), using serial as independant task
and changing ping to false :
bool WebSocket_Server::isConnected(){
if (_websocket_server) {
return _websocket_server->connectedClients(false) > 0;
}
return false;
}
I get the 15K report complete - very good catch
so may be doing a ping only every 50ms like you tested is the way to go : the max transmission would be during that period if baudrate is 250000 at 50ms = 1563 chars, and at 1000000 would be 6250 chars
Note: In 2.1 I did not used this API to check if connection is ok- so that explain also why issue did not popped
Note2: in 3.0 before client refactoring I did not used ->connectedClients(true) only ->connectedClients() so no issue neither
Which confirm the french quote: the best is the mortal enemy of good trying to check too much I broke the communications orz...
Hmm I modified the isConnected like this:
bool WebSocket_Server::isConnected(){
static uint64_t last_check = 0;
bool ping = false;
if (_websocket_server) {
if (millis() - last_check > 50) {
last_check = millis();
ping = false;
}
return _websocket_server->connectedClients(ping) > 0;
}
return false;
}
and still get few data lost (around 10~20) with WebUI only but they are significatives ones
increasing timeout to 100ms solved the issue
then I did same test but this time webui + telnet
then got data lost in webui and telnet:
few again but they are significatives
Telnet client is processed after WebSocket
So there is still some blocking area
Removing totally the ping check solved the issue (no red, differences are trailling spaces in webui) :
So I think removing the ping is the right solution because it introduce to much random delay on small messages and not using it did not caused issue in past
bool WebSocket_Server::isConnected(){
if (_websocket_server) {
return _websocket_server->connectedClients() > 0;
}
return false;
}
I have updated git - please let me know if now you still have some freeze or if everything is back to normal
I have added OUTPUT=PRINTER to solve several issue the
M117may no fit all targets fw, the space is handled differently in ESP3D 3.0 so[ESP111]M117\the space will be trimmed and M117 won't be recognize on some FW, in your case it is different you added IP: but that was not the original purpose
This "space" update is very welcome, using "\ " solves some of my issues.
I guess adding HEADER= as additionnal parameter is not big issue but I have hard time to understand the relationship between IP:192.168.2.5 and timestamp for printing ^_^
Adding the HEADER will allow for great flexibility, comparable to V2.1. The relationship is that if the ESP has gotten an IP address, then it might be online and the time might be available. So then the notification can be stamped with the actual time, and a print ETA can be calculated.
Is it possible for ESP3D 3.0 configure it to always respond with M117 ?
You mean [ESP111] ? that is weird if your are querying from telnet / websocket / webui, I prefere let user to do what it need and avoid most automatic choice that lead to complex code to handle all exeptions
I understand, that makes sense. The issue I have is this, the TFT boots up very fast, and is ready long before the ESP has an IP address (STA mode). So the TFT requesting an IP address from the ESP is useless at startup, but... the ESP3D reports when it receives an IP address, so if it would report "M117 IP:192.168.xxx.xxx" then the TFT can just wait for an IP address to be reported, and then request the time from the ESP. Perhaps it could be configured in the configuration.h.
The Charts panel gives me an additional empty chart below the "Entruder" and "Bed" chart.
That the sensors chart - you can hide it in settings if you do not have any
I cannot find where to disable the sensors chart. I only have the Extruders and Bed charts enable in the Charts Panel, I cannot find any reference to sensors under settings, except in the Extra Controls panel.
One more thing, I would recommend to use a delay for the flyovers of the buttons, no delay is a bit annoying.
I don't know if you also make the index.html, it holds this regex. I wonder which software is used to make the index.html
const o={T:[],R:[],B:[],C:[],P:[],M:[],L:[]},r=/(B|T|C|P|R)([\d]):([+|-]?[0-9].?[0-9]+|inf)? /([+]?[0-9]*.?[0-9]+)/g;
This matches "T:21.23 /25", but not "T:21.23/25" (without space before the slash).
Some clients don't have the space, so perhaps it's better to not require a space before the slash.
const o={T:[],R:[],B:[],C:[],P:[],M:[],L:[]},r=/(B|T|C|P|R)([\d]):([+|-]?[0-9].?[0-9]+|inf)?/([+]?[0-9]*.?[0-9]+)/g;
Humm I am not sure to follow :
there are 2 way to get the IP :
1 - automaticaly at start when wifi service start, be noted some people disable wifi at start so you user case may have an issue if you wait for this kind of information, you may be stuck
also there is no header for this announce
2 - [ESP111] it must be done manually and 2.1 allowed flexible header unlike 3.0 because so far no user case
You mention [ESP111] usage but your description looks more the automatic notification which has no header even in 2.1
Also I still do not understand the relation between IP and timestamp, you just need to know when esp board is starting ? or really when it got IP ?
Humm I am not sure to follow : there are 2 way to get the IP : 1 - automaticaly at start when wifi service start, be noted some people disable wifi at start so you user case may have an issue if you wait for this kind of information you may be stuck also there is header for this announce 2 - [ESP111] it must be done manually and 2.1 allowed flexible header unlike 3.0 because so far no user case
You are right of course. Currently I just wait and try to get the time after a delay, but it would be nice if the TFT could just wait for the IP address to arrive, then report the IP address to the user and get the time. Or even better, if the ESP would report the time when the time is available (configurable in configuration.h perhaps). Typically it's useful to provide info with a label or header so it's easier to pickup, rather than requiring an RegEx. In the TFT I cannot RegEx all input data, but I can scan for "IP:" or "DATE:" etc.
If Wifi is disabled then nothing can be done, that's up to the user.
You mention [ESP111] usage but your description looks more the automatic notification which has no header even in 2.1
Correct. Automatic is what I think would be nice, but currently it's done manually. This is under development...
Also I still do not understand the relation between IP and timestamp, you just need to know when esp board is starting ? or really when it got IP ?
The TFT doesn't know the time at startup, but when it gets an IP address from the ESP, then the ESP might know the time, so the TFT can request the time from the ESP, and the TFT can use the current time info (time stamp, provide ETA). So I care about when the ESP is online, which indicates it might know the time.
TFT can just wait for an IP address to be reported, and then request the time from the ESP. Perhaps it could be configured in the configuration.h.
be noted not everyone use the timestamp
I cannot find any reference to sensors under settings, except in the Extra Controls panel.
in charts panel ^_^
I don't know if you also make the index.html,
Yes I do the ESP3D-WEBUI
What client are you refering to that does not have space ? I am curious - I have tried to cover each FW properly
if the ESP would report the time when the time is available (configurable in configuration.h perhaps).
That looks more reasonable than trying complex things not related to time - but you need a trigger in M117 message to know it is time , or do you have a command that ESP should send including time ?
Or do you use json format to query ? which would be is easier to parse and get all data properly
be noted not everyone use the timestamp
Yes, of course. That's not a problem. In the TFT I just timestamp the notification if the time is available, otherwise I use relative print time to timestamp.
I don't have that "Show Sensors" option in the Settings/Charts, but I have an additional chart (without a label) in the Dashboard. Perhaps I disabled something in the configuration.
What tool do you use to create the index.html?
I have an old CNC machine (CNC3018), running GRBL, it reports the laser temperature. This machine is quite popular in China.
I think if you remove the space from the regular expression then both ways ("T:21.23 /25" and "T:21.23/25") will work.
you are not using the GRBL version of index.html ? I have a CNC3018 - but did not know it reported temperature can you elaborate ?
I use vscode : https://esp3d.io/esp3d-webui/v3.x/documentation/compilation/index.html
That looks more reasonable than trying complex things not related to time - but you need a trigger in M117 message to know it is time , or do you have a command that ESP should send including time ?
Or do you use json format to query ? which would be is easier to parse and get all data properly
I don't use json format. For the TFT is would be easiest to parse something like:
IP:192.168.1.123
DATE:2024-02-05 17:09:41
Similar to how Marlin works, just add "T:" in front of the temp, or B: C: etc.
There are many DIY kits for the CNC3018, this one can monitor laser temperature and motor temperature.
It actually waits for the laser to warm up before starting. I believe it runs GRBL 0.9?. I is probably customized.
so you with grbl should use this webui not 3d printer one
About your request I have a proposal :
if there is a setting PRINTER_HAS_DISPLAY in configuration.h, it will enable the command [ESP212]<text>
In this text you will be able to use some variable like %ESP_IP% for local IP, %ESP_NAME%, for local name, '%ESP_DATETIME%' for ESP date time
Also I will add a hook when IP is get to process a command and another one when time is get to process a another command, both defined in configuration.h, so you can use [ESP212]IP:%ESP_IP% that wil generate M117 IP:192.168.2.50 and [ESP212]DATE:%ESP_DATETIME% that will generate M117 DATE:2024-01-05 17:40:00
Does it fit your needs ?
But it will only work with 3D Printers because grbl does not support M117
About your request I have a proposal : if there is a setting PRINTER_HAS_DISPLAY in configuration.h, it will enable the command
[ESP212]<text>In this text you will be able to use some variable like%ESP_IP%for local IP,%ESP_NAME%, for local name, '%ESP_DATETIME%' for ESP date time Also I will add a hook when IP is get to process a command and another one when time is get to process a another command, both defined in configuration.h, so you can use[ESP212]IP:%ESP_IP%that wil generateM117 IP:192.168.2.50and[ESP212]DATE:%ESP_DATETIME%that will generateM117 DATE:2024-01-05 17:40:00Does it fit your needs ?
That would be fantastic, it's super flexible, and makes integration very easy, you have my vote :D
But it will only work with 3D Printers because grbl does not support M117
For the CNC system my only problem is that I need to modify the RegEx for the temperature readings to be read. The system doesn't have a display.
Do you have any idea why I have this extra empty chart that I cannot disable?
I'm not sure if you still maintain V2.1, but I noticed something odd.
In syncwebserver.cpp '\r' is filtered out: current_buffer.replace("\r","");
I believe this operation should be moved outside of the while loop (3 lines above), it does not need to be repeated for every line ending on '\n'.
That would be fantastic, it's super flexible, and makes integration very easy, you have my vote :D
Ok should be done in couple of days
For the CNC system my only problem is that I need to modify the RegEx for the temperature readings to be read
I still do not understand you mentionned you use grbl but you seems using 3D printer webui ... why ?
Do you have any idea why I have this extra empty chart that I cannot disable?
Nope, I cannot reproduce, may be it is linked to the way you use the webui or the preferences.json is corrupted - delete it
I'm not sure if you still maintain V2.1,
Yes only for critical bugs, all new features enhancement go to 3.0, optimization like you suggest are neither done anymore but checked on 3.0 instead
That would be fantastic, it's super flexible, and makes integration very easy, you have my vote :D
Ok should be done in couple of days
For the CNC system my only problem is that I need to modify the RegEx for the temperature readings to be read
I still do not understand you mentionned you use grbl but you seems using 3D printer webui ... why ?
Yes, I should change that. I was just checking out the status of V3.0 and didn't look to closely into it.
Do you have any idea why I have this extra empty chart that I cannot disable?
Nope, I cannot reproduce, may be it is linked to the way you use the webui or the preferences.json is corrupted - delete it
Deleting the preferences did the trick, thanks!
please give a try to this test branch : https://github.com/luc-github/ESP3D/tree/ESP214-and-hooks
Command is [ESP212] to send to screen
Configuration.h has the hooks enabled for IP and for DATETIME
On my side it looks ok
please give a try to this test branch : https://github.com/luc-github/ESP3D/tree/ESP214-and-hooks
That is fast, I certainly will, thanks.
I checked if if I could find the reason for my charts issues.
I think I found a typo in Charts.js (should probably be 0, 1, 2, but is 0, 1, 1)
charts[0].ref = useRef(null)
charts[1].ref = useRef(null)
charts[1].ref = useRef(null)
It seems the charts are supposed to have different colors, but currently they are all pink (bed and extruder).
No very important, but there are several mistakes in the comments for "chartColors" (Charts.js)
const chartColors = [
"255,128,128", //pink
"0,0,255", //dark blue
"0,128,0", //dark green
"198,165,0", //gold
"255,0,0", //red
"0,0,128", //blue
"128,255,128", //light green
"255,128,0", //orange
"178,0,255", //purple
"0,128,128", //green blue
"128,128,0", //kaki
"128,128,128", //grey
"0,0,0", //purple
]
const chartColors = [
"255,128,128", //pink
"0,0,255", //blue
"0,128,0", //dark green
"198,165,0", //gold
"255,0,0", //red
"0,0,128", //dark blue
"128,255,128", //light green
"255,128,0", //orange
"178,0,255", //purple
"0,128,128", //green blue
"128,128,0", //olive
"128,128,128", //grey
"0,0,0", //black
]
I remember I did on purpose - the green color was not great alone so I kept the blue for first third chart and I did not spent time to optimize the color rendering to be honest
yes comment are copy and paste issues (fixed in code now)
Confirmed, the freeze issue is fixed by patching "WebSocket_Server::isConnected()".
The ESP3D response in the browser is now much faster and more consistent.
I tested it on an ESP32 and ESP32S2.
Thanks for the great support!
Should I close this topic and open a Feature Request for the "hooks"?
BTW: In esp3d/src/modules/notifications/notifications_service.cpp there is a typo on line 681, I guess it should be _token2 = "";
I enabled some compiler warnings, but to my surprise this issue is not pointed out.
Confirmed, the freeze issue is fixed by patching "WebSocket_Server::isConnected()".
Cool then I close issue
Thanks for the great support!
Well you did most of the work and investigation - thanks to you ^_^
Should I close this topic and open a Feature Request for the "hooks"?
No need just comment on PR
BTW: In esp3d/src/modules/notifications/notifications_service.cpp there is a typo on line 681, I guess it should be _token2 = "";
Yes again a bad copy and paste - orz .... fixed now in coming PR thank you
I enabled some compiler warnings, but to my surprise this issue is not pointed out.
Because doing twice the same command is not an error it is more a logic error - and that will be only seen if you ask an IA to review the code which I do not do usually, may be I should but I lack of time to review all ^_^
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.