s00500 / ESPUI

A simple web user interface library for ESP32 and ESP8266

Home Page:https://valencia.lbsfilm.at/midterm-presentation/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

RELOAD UI request from Server to browser has no effect

MartinMueller2003 opened this issue · comments

Issuing ESPUI.jsonReload has no effect.

I have instrumented the code to print a message when the reload command is issued and again for every call to jsonDom. After the reload is issued, there are no calls to jsonDom. Clicking on the reload page button on the browser does show the expected calls to jsonDom.

As a secondary issue, issuing jsonDom(0) from a task results in the first dom transfer to happen but no subsequent transfers are requested by the browser.

There are no errors being reported on the console.
There are no other calls to send anything to the browser immediately following the call to jsonReload

More digging on this issue. It looks like the problem is an overrun condition in the web socket egress queue. When a large number of items get enqueued in a single callback, some outgoing messages are silently discarded. I added code to print when a client can and cannot take more requests and the reload and jsonDom requests fall solidly in the discard space.

client 1 took data
client 1 took data
.... (~40 messages)
client 1 took data
client 1 took data
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:357: Start ------
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:362: Choice value: default ------
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:363: Text value: s1 ------
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:373: Cant do anything to the default entry ------
client 1 took data
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:417: Name exists and it is not the selected name ------
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:418: Key: 's1' ------
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:431: CreateAllowed ------
client 1 took data
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:433: DeleteAllowed ------
client 1 took data
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:435: UpdateAllowed ------
client 1 cannot take more data
------ CbTextChange:src/Controllers/ControllerFPPDSequences.cpp:438: End ------
------ CbButtonCreate:src/Controllers/ControllerFPPDSequences.cpp:226: Set the Select List to s1 ------
client 1 cannot take more data
------ CbButtonCreate:src/Controllers/ControllerFPPDSequences.cpp:229: Activate ------
client 1 cannot take more data

And on. The condition clears as soon as we return from the callback. I am going to leave the detection code and add a mechanism that can be used to trigger a future jsonDom. I am also going to look at deferring the updates until the end of the callback and then bulk upload them using a mechanism similar to the jsonDom exchange but only load controls that have an update flag set.

If anyone has a better idea, please ping me and we can discuss.

Sounds like a good plan, how many controls do you have in your setup ?

Right... that is quite an amount :-)

This is related to a general issue with AsyncWebSocket. It has a hardcoded limit of 8 queued messages on 8266 and 32 on ESP32. Any more queued messages are silently discarded. The AsyncWebSocket class provides a member function queueIsFull() that we could use to ensure that there is space in the queue...but currently we don't.

A bit of a re-architecture would be required to properly fix this. The only operation likely to ever hit it is jsonDom, so it could perhaps be make a stateful operation (called scheduleResend() or something) which then requires a checkForResend() call in loop() or similar. Then it could use queueIsFull() to only schedule as many messages can be sent.

The problem with this would be the requirement to now put code into loop() that we didn't have before. Unless anyone has any thoughts?

The problem with this would be the requirement to now put code into loop() that we didn't have before. Unless anyone has any thoughts?

Personally I don't have any issue with a service routine in the main loop. But tor backwards compatibility this would need to be an optional feature. That is to say, default behavior should be as before so that existing projects don't break after upgrading ESPUI. The new queue feature could be enabled by setup code.

Any chance this issue is involved with the random web socket disconnects? Would be great to resolve that nuisance too.

  • Thomas

It is unfortunate that this is the only way to do it.... on the esp32 we could theoretically use an rtos task, but not on the esp8266... and then we are still in arduino world so people are sometimes going to use blocking delays anyways....

For now I agree with @thomastech, I think this can be added without breaking existing things, when it gets called for the first time we can set a flag to change the global behavior....

I am working something out that bulks up requests but I need a trigger to send. jsonDom is the only function that is not significantly impacted by this. I am at a point where I need to trigger a 100ms timer to allow messages to bulk up that are NOT the result of a callback. I have all messages generated while in a callback working. If we had a regular ping running then I would have the last trigger we need.

I think I have found a timer that will work well. I am going to have the js client output a single "update" (this is a new message) request 500ms after it sends a control change to the server. If there are multiple messages then on any client send or receive operation, the timer is refreshed. That means a single extra kick will happen if there are no messages between client and server for 500ms and then nothing more.

Just be sure that the chosen solution does not adversely affect concurrent browser sessions. I sometimes have a session or two on desktops while also accessing via mobile device.

  • Thomas

By using a timer on each browser and having the browser requesting an update I get each browser updated.

OK. The timer mechanism works which proves the theory, but I hate timers. I have a few ideas still percolating about. I am still going for 100% backwards compatibility (aka no existing user needs to change any code to continue to use the library) and I hate to use an ESP timer because there is no such thing as a shared timer resource in the base OS on the ESP8266 and it would require a new library to be added to the ESP8266 builds, making these changes less than 100% backwards compatible.

At the moment I am looking at a modification of my current idea:

  1. When the egress queue is empty create an update and send it (it will sit in the queue until the task ends)
  2. If the egress queue is not empty, save the updates until later
  3. client receives the update and after some ms pokes the server for more updates
  4. server sends the rest of the updates in bulk messages

FYI: I have modified how bulk messages are generated and I fill the json doc structure to capacity. I am seeing up to 51 controls in a single bulk message so the overall effect of these changes is to reduce traffic load on the network when compared to the current release. The new mechanism is fully adaptive and can handle a situation where a single control fills the entire structure..

I dumped the timer mechanism and went to a small trigger mechanism with a per client state machine that manages updates and keeps the number of outgoing ws messages on adds and updates very small. PR has been created.

This is resolved by the restructuring.

This is now fixed. The root issue was caused by too many messages going to the UI which overflowed the OS message queue.