RecursionError: maximum recursion depth exceeded
moepnse opened this issue · comments
Hello!
With the latest development snapshot, I get recurring recursion errors after a long time. As far as I can see there is no recursion. Since my project code is very large, I will try to write an easier to understand issue demo that reproduces the error in the next few days.
...
if self._update_system_health_subscription is None:
self._update_system_health_subscription = window.session.subscribe(f'{target_system}.gemini', self.update_system_health_plots)
...ts3>ack update_system_health_plots
experiment.epy
235: self._update_system_health_subscription = window.session.subscribe(f'{target_system}.gemini', self.update_system_health_plots)
237: client def update_system_health_plots(self, data, *args, **kwargs):
Traceback (most recent call last):
File "?brython-version=snapshot", line 18, in onopen
import elephas
File "VFS.elephas/__init__.py", line 68, in <module>
from . import ui
File "VFS.elephas.ui/__init__.py", line 6, in <module>
from . import splash
File "VFS.elephas.ui.uiplugins.py", line 44, in <module>
browser.aio.run(load_ui_plugins())
File "VFS.elephas.ui.uiplugins.py", line 43, in load_ui_plugins
UIPlugin.plugins_loaded()
File "/py/elephas/plugins/ui/0_base_component.py", line 185, in register
BaseComponent.register()
File "/py/elephas/plugins/ui/0_base_component.py", line 129, in register
Splash.hide_splash()
File "VFS.stermi.router.py", line 177, in event_handler
console.debug("No class handler found!")
File "/py/elephas/plugins/ui/app.py", line 144, in load_object
aio.run(_load_object(self, ev, router, params, query))
File "/py/elephas/plugins/ui/app.py", line 105, in _load_object
router.route(template, Default(attrs))
File "VFS.stermi.router.py", line 77, in route
history.append(page)
File "/py/elephas/plugins/ui/page.py", line 15, in connectedCallback
self.init()
File "/py/elephas/plugins/ui/0_base_component.py", line 90, in connectedCallback
self._initialized = True
File "VFS.stermi.page.py", line 65, in connectedCallback
self._initialized=True
File "/py/elephas/plugins/ui/0_base_component.py", line 90, in connectedCallback
self._initialized = True
File "VFS.stermi.page_content.py", line 73, in connectedCallback
self._initialized=True
File "VFS.stermi.page_content.py", line 27, in connectedCallback
self._append_childs()
File "VFS.stermi.page_content.py", line 64, in _append_childs
del self._page_content_elements
File "/py/elephas/plugins/ui/tabs.py", line 49, in connectedCallback
STab.connectedCallback(self)
File "VFS.stermi.tabs.py", line 126, in connectedCallback
self._initialized=True
File "VFS.stermi.page_content.py", line 27, in connectedCallback
self._append_childs()
File "VFS.stermi.page_content.py", line 64, in _append_childs
del self._page_content_elements
File "VFS.stermi.page_content.py", line 46, in append_child
BaseComponent.append_child(self,element)
File "VFS.stermi.base_component.py", line 401, in append_child
child_container.append_child(element)
File "VFS.stermi.list.py", line 198, in append_child
self._ul.appendChild(li)
File "/py/elephas/plugins/ui/select.py", line 42, in connectedCallback
aio.run(self.load_items())
File "/py/elephas/plugins/ui/select.py", line 88, in load_items
self._load_in_progress = False
File "VFS.ts3.experiment.py", line 92, in update_system_health_plots
self._system_health_first_plot=False
File "VFS.stermi.time_series.py", line 125, in new_plot
render(self._container_div,data,layout)
File "VFS.stermi.time_series.py", line 69, in render
window.Plotly.newPlot(
File "VFS.ts3.experiment.py", line 132, in update_system_health_plots
self._ts_partitions.extend_traces(data,plot_indexes_to_modify)
File "VFS.stermi.time_series.py", line 119, in relayout
window.Plotly.relayout(self._container_div,update)
File "VFS.ts3.experiment.py", line 132, in update_system_health_plots
self._ts_partitions.extend_traces(data,plot_indexes_to_modify)
File "VFS.stermi.time_series.py", line 119, in relayout
window.Plotly.relayout(self._container_div,update)
File "VFS.ts3.experiment.py", line 132, in update_system_health_plots
self._ts_partitions.extend_traces(data,plot_indexes_to_modify)
File "VFS.stermi.time_series.py", line 119, in relayout
window.Plotly.relayout(self._container_div,update)
File "VFS.ts3.experiment.py", line 132, in update_system_health_plots
self._ts_partitions.extend_traces(data,plot_indexes_to_modify)
File "VFS.stermi.time_series.py", line 119, in relayout
window.Plotly.relayout(self._container_div,update)
File "VFS.ts3.experiment.py", line 132, in update_system_health_plots
self._ts_partitions.extend_traces(data,plot_indexes_to_modify)
File "VFS.stermi.time_series.py", line 119, in relayout
window.Plotly.relayout(self._container_div,update)
...
File "VFS.ts3.experiment.py", line 132, in update_system_health_plots
self._ts_partitions.extend_traces(data,plot_indexes_to_modify)
File "VFS.stermi.time_series.py", line 119, in relayout
window.Plotly.relayout(self._container_div,update)
File "VFS.ts3.experiment.py", line 98, in update_system_health_plots
start_time=datetime.fromtimestamp(start_time,tz=timezone.utc).strftime(format_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "VFS._pydatetime.py", line 1806, in fromtimestamp
return cls._fromtimestamp(timestamp,tz is not None,tz)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "VFS._pydatetime.py", line 1795, in _fromtimestamp
result=tz.fromutc(result)
^^^^^^^^^^^^^^^^^^
File "VFS._pydatetime.py", line 2417, in fromutc
return dt+self._offset
^^^^^^^^^^^^^^^
File "VFS._pydatetime.py", line 2216, in __add__
delta=timedelta(self.toordinal(),
^^^^^^^^^^^^^^^^
File "VFS._pydatetime.py", line 1103, in toordinal
return _ymd2ord(self._year,self._month,self._day)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "VFS._pydatetime.py", line 73, in _ymd2ord
dim=_days_in_month(year,month)
^^^^^^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded
brython.js:10974:56
flush http://localhost:8082/brython/snapshot/brython.js:10974
method http://localhost:8082/brython/snapshot/brython.js:1924
show_error http://localhost:8082/brython/snapshot/brython.js:4541
handle_error http://localhost:8082/brython/snapshot/brython.js:4545
<anonym> debugger eval code:1Thanks in advance!
The issue demo below, will draw the plots flawlessly and after a while start throwing the above error. So be patient. :-)
<!DOCTYPE html>
<html>
<head>
<!-- Required meta tags-->
<meta charset="utf-8">
<title>Issue Demo</title>
<!-- Brython -->
<script src="https://raw.githack.com/brython-dev/brython/master/www/src/brython.js"></script>
<script src="https://raw.githack.com/brython-dev/brython/master/www/src/brython_stdlib.js"></script>
<script src="https://cdn.plot.ly/plotly-1.57.1.min.js"></script>
<script>
var org_locale = Plotly.d3.locale;
function tformat(d) {
sec = d % (24 * 3600);
hour = Math.floor(sec / 3600);
sec = sec % 3600;
min = Math.floor(sec / 60);
sec = sec % 60;
formated_str = String(hour).padStart(2, '0') + ":" + String(min).padStart(2, '0') + ":" + String(sec).padStart(2, '0');
return formated_str;
}
function bytesFormat(bytes) {
if (!+bytes) return '0 Bytes'
const k = 1024
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']
const i = Math.floor(Math.log(bytes) / Math.log(k))
return `${parseFloat((bytes / Math.pow(k, i)).toFixed(2))} ${sizes[i]}`
}
Plotly.d3.locale = (locale) => {
var result = org_locale(locale);
var org_number_format = result.numberFormat;
result.numberFormat = (format) => {
if (format == 'tformat') {
return tformat;
} else if (format == "bformat") {
return bytesFormat;
}
return org_number_format(format);
}
return result;
}
</script>
<script>
function subscribe(uri, callback) {
function _call() {
console.debug("calling callback...")
callback({'cpu': [16.4, 9.2, 30.8, 12.3], 'mem': 46.3, 'partitions': {'C:\\': {'total': 982636294144, 'used': 239104602112}, 'D:\\': {'total': 16241389568, 'used': 14304120832}}})
}
setInterval(_call, 1000)
}
</script>
<script type="text/python">
from browser import webcomponent, html, window, console, document
class BaseComponent:
_registry = []
_initialized = False
_logic_obj = None
_is_container: bool = False
@staticmethod
def un_camel(word: str) -> str:
upper_chars: str = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
last_char: str = word[0]
output: list = [last_char.lower()]
for c in word[1:]:
if c == "_":
output.append("-")
continue
if c in upper_chars:
if last_char not in upper_chars:
output.append('-')
output.append(c.lower())
else:
output.append(c)
last_char = c
return "".join(output)
@classmethod
def __init_subclass__(cls, **kwargs):
BaseComponent._registry.append(cls)
@classmethod
def remove_from_registry(cls, component):
print(cls._registry, component)
if component in cls._registry:
cls._registry.remove(component)
@classmethod
def register(cls):
registry = cls._registry
for web_component in registry:
web_component_name = cls.un_camel(web_component.__name__)
component_name = f"ui-{web_component_name}"
console.debug(f"registering web component {web_component} as {component_name}...")
webcomponent.define(component_name, web_component)
def register_logic_class(self, cls):
self._logic_class = cls
logic_obj = cls(self)
self._logic_obj = logic_obj
def render(target, data, layout):
window.Plotly.newPlot(
target,
data,
layout
)
class TimeSeries(BaseComponent):
def connectedCallback(self):
if not self._initialized:
# container div
self._container_div = div = document.createElement("div")
self.appendChild(div)
_locals = {}
config = self.querySelector("UI-TIME-SERIES-CONFIG")
if config:
bry_src = config.text
_globals = {}
exec(bry_src, _globals, _locals)
data = _locals.get("data", [])
layout = _locals.get("layout", [])
render(self._container_div, data, layout)
self._initialized = True
def restyle(self, restyle_data, trace_indexes=None):
console.debug(restyle_data)
window.Plotly.restyle(self._container_div, restyle_data, trace_indexes)
def relayout(self, update):
window.Plotly.relayout(self._container_div, update)
def resize(self,):
window.Plotly.Plots.resize(self._container_div)
def new_plot(self, data, layout):
render(self._container_div, data, layout)
def extend_traces(self, data, plot_indexes_to_modify):
window.Plotly.extendTraces(self._container_div, data, plot_indexes_to_modify)
def update(self, data):
window.Plotly.update(self._container_div, data)
class TimeSeriesConfig(BaseComponent):
pass
class Page(BaseComponent):
pass
class Logic(BaseComponent):
dependencies = ("Page", "PageContent")
def connectedCallback(self):
if not self._initialized:
def register_logic_class(cls):
nonlocal parent
if hasattr(parent, "register_logic_class"):
console.debug("registering logic class:", cls)
parent.register_logic_class(cls)
else:
console.error(f"{parent} has no logic class support")
parent = self.parentElement
bry_src = """
"""
bry_src += self.text
try:
exec(bry_src, {"parent": parent, "register_logic_class": register_logic_class})
except Exception as e:
console.error(e)
self._initialized = True
BaseComponent.register()
</script>
</head>
<body onload="brython({debug: 2})">
<ui-page>
<ui-time-series name="ts_cpu"></ui-time-series>
<ui-time-series name="ts_mem"></ui-time-series>
<ui-time-series name="ts_partitions"></ui-time-series>
<ui-logic style="display: none;">
import time
from datetime import datetime, timezone
import math
from browser import window, console, aio
class Demo:
def __init__(self, parent):
self._update_plot_subscription = None
self._update_system_health_subscription = None
self._parent = parent
# vvvv - system healt plots
self._ts_cpu = self._parent.querySelector("ui-time-series[name='ts_cpu']")
self._ts_mem = self._parent.querySelector("ui-time-series[name='ts_mem']")
self._ts_partitions = self._parent.querySelector("ui-time-series[name='ts_partitions']")
if self._ts_cpu and self._ts_mem:
aio.run(self.init_cpu_n_mem_plot())
# ^^^^
async def init_cpu_n_mem_plot(self, target_system=None):
console.debug("init_cpu_n_mem_plot")
target_system = ""
color_palette: list = [ "orange", "red",
"green", "blue",
"purple", "olive",
"tomato", "maroon",
"brown", "navy",
"orchid", "silver",
"skyeblue", "palegreen"]
self._color_palette = color_palette.copy()
self._layout = layout = {
'title': 'Live Stream',
'xaxis': {
'title': "time"
},
'yaxis': {
'title': "values"
}
}
self._system_health_first_plot = True
console.debug(f'{target_system}.gemini')
if self._update_system_health_subscription is None:
self._update_system_health_subscription = window.subscribe(f'{target_system}.gemini', self.update_system_health_plots)
def update_system_health_plots(self, data, *args, **kwargs):
cpu, mem, partitions = data["cpu"], data["mem"], data["partitions"].to_dict()
cpu_count = len(cpu)
partition_count = len(partitions)
format_str = "%Y-%m-%d %H:%M:%S"
if self._system_health_first_plot == True:
ts = time.time()
#t = datetime.datetime.utcfromtimestamp(t).strftime(format_str)
t = datetime.fromtimestamp(ts, tz=timezone.utc).strftime(format_str)
data_x, data_y, mem_x, mem_y, partition_x = [t]*cpu_count, cpu, [t], [mem], [t]*partition_count
# vvvv - usage per cpu core
columns = 2
rows = math.ceil(cpu_count / columns)
height = cpu_count * 200
self._ts_cpu._container_div.style.height = f"{height}px"
self._cpu_layout = cpu_layout = {"grid": {
'title': 'CPU usage',
'rows': rows,
'columns': columns,
'pattern': 'independent',
'roworder': 'bottom to top'}
}
color_palette = self._color_palette
traces: list = []
for x, y, cpu_nr, color, i in zip(data_x, data_y, range(cpu_count), color_palette, range(1, cpu_count+1)):
trace = {
'type': "scatter",
#'mode': "lines",
'name': f"core {cpu_nr}",
'line': {'color': color},
'fill': 'tozeroy',
'fillcolor': color,
'x': [x],
'y': [y],
'xaxis': f'x{i}',
'yaxis': f'y{i}'
}
traces.append(trace)
console.debug("traces:", traces)
self._ts_cpu.new_plot(traces, cpu_layout)
# ^^^^
# vvvv - Memory usage
mem_layout = {
'title': 'Memory Usage',
'xaxis': {
'title': "time"
},
'yaxis': {
'title': "Percent"
}
}
mem_traces = [{
'type': "scatter",
#'mode': "lines",
'name': "Memory usage",
'line': {'color': "red"},
'fill': 'tozeroy',
'fillcolor': 'red',
'x': mem_x,
'y': mem_y,
}]
self._ts_mem.new_plot(mem_traces, mem_layout)
# ^^^^
# vvvv - partitions
columns = 2
rows = math.ceil(partition_count / columns)
height = partition_count * 200
self._ts_partitions._container_div.style.height = f"{height}px"
self._partition_layout = partition_layout = {"grid": {
'title': 'Partition usage',
'rows': rows,
'columns': columns,
'pattern': 'independent',
'roworder': 'bottom to top'}
}
color_palette = self._color_palette
traces: list = []
for x, partition_mountpoint, partition_nr, color, i in zip(partition_x, partitions, range(partition_count), color_palette, range(1, partition_count+1)):
partition = partitions[partition_mountpoint]
y = partition["used"]
trace = {
'type': "scatter",
#'mode': "lines",
'name': partition_mountpoint,
'line': {'color': color},
'fill': 'tozeroy',
'fillcolor': color,
'x': [x],
'y': [y],
'xaxis': f'x{i}',
'yaxis': f'y{i}'
}
traces.append(trace)
console.debug("traces:", traces)
self._ts_partitions.new_plot(traces, partition_layout)
# ^^^^
self._system_health_first_plot = False
else:
t = time.time()
start_time = t - 60 * 5
end_time = t
start_time = datetime.fromtimestamp(start_time, tz=timezone.utc).strftime(format_str)
end_time = datetime.fromtimestamp(end_time, tz=timezone.utc).strftime(format_str)
t = datetime.fromtimestamp(t, tz=timezone.utc).strftime(format_str)
# vvvv- cpu usage
data = {"x": [[t]]*cpu_count, "y": [[c] for c in cpu]}
plot_indexes_to_modify = list(range(len(data["y"])))
cpu_layout = self._cpu_layout
for core in range(1, cpu_count+1):
if core == 1:
xaxis = "xaxis"
yaxis = "yaxis"
else:
xaxis = "xaxis" + str(core)
yaxis = "yaxis" + str(core)
cpu_layout[xaxis] = {
"type": 'date',
"range": [start_time, end_time],
'title': f'CPU Usage (core {core})',
}
cpu_layout[yaxis] = {
'title': "Percent",
"range": [0, 100]
}
self._ts_cpu.relayout(cpu_layout)
self._ts_cpu.extend_traces(data, plot_indexes_to_modify)
# ^^^^
# vvvv - memory usage
mem_layout = {
"xaxis": {
"type": 'date',
"range": [start_time, end_time],
'title': 'Memory Usage',
},
'yaxis': {
'title': "Percent",
"range": [0, 100]
}
}
self._ts_mem.relayout(mem_layout)
self._ts_mem.extend_traces({"x": [[t]], "y": [[mem]]}, [0])
# ^^^^
# vvvv- partition usage
data = {"x": [[t]]*partition_count, "y": [[p["used"]] for mp, p in partitions.items()]}
plot_indexes_to_modify = list(range(len(data["y"])))
partition_layout = self._partition_layout
for i, mountpoint in zip(range(1, partition_count+1), partitions):
partition = partitions[mountpoint]
if i == 1:
xaxis = "xaxis"
yaxis = "yaxis"
else:
xaxis = "xaxis" + str(i)
yaxis = "yaxis" + str(i)
partition_layout[xaxis] = {
"type": 'date',
"range": [start_time, end_time],
'title': mountpoint,
}
partition_layout[yaxis] = {
'title': "Used",
"range": [0, partition["total"]],
"tickformat": "bformat"
}
self._ts_partitions.relayout(partition_layout)
self._ts_partitions.extend_traces(data, plot_indexes_to_modify)
# ^^^^
register_logic_class(Demo)
</ui-logic>
</ui-page>
</body>
</html>The error occurs after 496 seconds.
I have tested the old 3.11 version and the error does not occur there.
According to my research, the bug is caused by this change: d9c69bd#diff-766a8e6709ecee5361c7b50b15c4290cab5e2d386ef1512be99eda2ad5521753L153-R162
I think the problem comes from the fact that the frame object is not copied, but referenced. I will take a closer look at this later.
...
if(jsobj instanceof Promise || typeof jsobj.then == "function"){
//return jsobj.then(x => jsobj2pyobj(x)).catch($B.handle_error)
// save frame obj and restore it on resolve / reject
var save_frame_obj = $B.frame_obj // Creates a reference, not a copy
jsobj.$frame_obj = $B.frame_obj
return jsobj.then(function(x){
$B.frame_obj = save_frame_obj
return jsobj2pyobj(x)
}).catch(function(err){
$B.frame_obj = save_frame_obj
throw $B.exception(err)
})
}
...Looks like my guess was wrong. The function relayout of Plotly returns a Promise object, which is not expected in my example and leads to the error after a while.
Unfortunately I don't have a solution yet.
Hello !
I don't think the issue is related to Brython. Previous releases might have shadowed the consequences, but the cause is the program logic, where asynchronous and synchronous code are mixed in a way that is likely to cause errors.
Here the root of the problem is that Plotly.relayout returns a Promise, and it is not handled as such : it is called (and not awaited) inside update_system_health_plots() through calls to method .relayout() of various objects, and update_system_health_plots itself is synchronous.
Moreover, update_system_health_plots is called repeatedly inside a setInterval, which itself calls (and not awaits) the function.
I modified your code in a few places and it now works without errors for hundreds of cycles (I stopped around 1500):
<!DOCTYPE html>
<html>
<head>
<!-- Required meta tags-->
<meta charset="utf-8">
<title>Issue Demo</title>
<!-- Brython -->
<script src="https://raw.githack.com/brython-dev/brython/master/www/src/brython.js"></script>
<script src="https://raw.githack.com/brython-dev/brython/master/www/src/brython_stdlib.js"></script>
<script src="https://cdn.plot.ly/plotly-1.57.1.min.js" crossorigin="anonymous"></script>
<script>
var org_locale = Plotly.d3.locale;
function tformat(d) {
sec = d % (24 * 3600);
hour = Math.floor(sec / 3600);
sec = sec % 3600;
min = Math.floor(sec / 60);
sec = sec % 60;
formated_str = String(hour).padStart(2, '0') + ":" + String(min).padStart(2, '0') + ":" + String(sec).padStart(2, '0');
return formated_str;
}
function bytesFormat(bytes) {
if (!+bytes) return '0 Bytes'
const k = 1024
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']
const i = Math.floor(Math.log(bytes) / Math.log(k))
return `${parseFloat((bytes / Math.pow(k, i)).toFixed(2))} ${sizes[i]}`
}
Plotly.d3.locale = (locale) => {
var result = org_locale(locale);
var org_number_format = result.numberFormat;
result.numberFormat = (format) => {
if (format == 'tformat') {
return tformat;
} else if (format == "bformat") {
return bytesFormat;
}
return org_number_format(format);
}
return result;
}
</script>
<script type="text/python">
from browser import webcomponent, html, window, console, document, timer
class BaseComponent:
_registry = []
_initialized = False
_logic_obj = None
_is_container: bool = False
@staticmethod
def un_camel(word: str) -> str:
upper_chars: str = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
last_char: str = word[0]
output: list = [last_char.lower()]
for c in word[1:]:
if c == "_":
output.append("-")
continue
if c in upper_chars:
if last_char not in upper_chars:
output.append('-')
output.append(c.lower())
else:
output.append(c)
last_char = c
return "".join(output)
@classmethod
def __init_subclass__(cls, **kwargs):
BaseComponent._registry.append(cls)
@classmethod
def remove_from_registry(cls, component):
print(cls._registry, component)
if component in cls._registry:
cls._registry.remove(component)
@classmethod
def register(cls):
registry = cls._registry
for web_component in registry:
web_component_name = cls.un_camel(web_component.__name__)
component_name = f"ui-{web_component_name}"
console.debug(f"registering web component {web_component} as {component_name}...")
webcomponent.define(component_name, web_component)
def register_logic_class(self, cls):
self._logic_class = cls
logic_obj = cls(self)
self._logic_obj = logic_obj
def render(target, data, layout):
window.Plotly.newPlot(
target,
data,
layout
)
class TimeSeries(BaseComponent):
def connectedCallback(self):
if not self._initialized:
# container div
self._container_div = div = document.createElement("div")
self.appendChild(div)
_locals = {}
config = self.querySelector("UI-TIME-SERIES-CONFIG")
if config:
bry_src = config.text
_globals = {}
exec(bry_src, _globals, _locals)
data = _locals.get("data", [])
layout = _locals.get("layout", [])
render(self._container_div, data, layout)
self._initialized = True
def restyle(self, restyle_data, trace_indexes=None):
console.debug(restyle_data)
window.Plotly.restyle(self._container_div, restyle_data, trace_indexes)
def relayout(self, update):
return window.Plotly.relayout(self._container_div, update)
def resize(self,):
window.Plotly.Plots.resize(self._container_div)
def new_plot(self, data, layout):
render(self._container_div, data, layout)
def extend_traces(self, data, plot_indexes_to_modify):
window.Plotly.extendTraces(self._container_div, data, plot_indexes_to_modify)
def update(self, data):
window.Plotly.update(self._container_div, data)
class TimeSeriesConfig(BaseComponent):
pass
class Page(BaseComponent):
pass
class Logic(BaseComponent):
dependencies = ("Page", "PageContent")
def connectedCallback(self):
if not self._initialized:
def register_logic_class(cls):
nonlocal parent
if hasattr(parent, "register_logic_class"):
console.debug("registering logic class:", cls)
parent.register_logic_class(cls)
else:
console.error(f"{parent} has no logic class support")
parent = self.parentElement
bry_src = """
"""
bry_src += self.text
try:
exec(bry_src, {"parent": parent, "register_logic_class": register_logic_class})
except Exception as e:
console.error(e)
self._initialized = True
BaseComponent.register()
</script>
</head>
<body onload="brython({debug: 2})">
<ui-page>
<ui-time-series name="ts_cpu"></ui-time-series>
<ui-time-series name="ts_mem"></ui-time-series>
<ui-time-series name="ts_partitions"></ui-time-series>
<ui-logic style="display: none;">
import time
from datetime import datetime, timezone
import math
from browser import window, console, aio
async def subscribe(uri, callback):
async def _call():
print('calling callback...')
await callback({'cpu': [16.4, 9.2, 30.8, 12.3], 'mem': 46.3, 'partitions': {'C:\\': {'total': 982636294144, 'used': 239104602112}, 'D:\\': {'total': 16241389568, 'used': 14304120832}}})
while True:
await _call()
await aio.sleep(0.1)
class Demo:
def __init__(self, parent):
self._update_plot_subscription = None
self._update_system_health_subscription = None
self._parent = parent
# vvvv - system healt plots
self._ts_cpu = self._parent.querySelector("ui-time-series[name='ts_cpu']")
self._ts_mem = self._parent.querySelector("ui-time-series[name='ts_mem']")
self._ts_partitions = self._parent.querySelector("ui-time-series[name='ts_partitions']")
if self._ts_cpu and self._ts_mem:
aio.run(self.init_cpu_n_mem_plot())
# ^^^^
async def init_cpu_n_mem_plot(self, target_system=None):
console.debug("init_cpu_n_mem_plot")
target_system = ""
color_palette: list = [ "orange", "red",
"green", "blue",
"purple", "olive",
"tomato", "maroon",
"brown", "navy",
"orchid", "silver",
"skyeblue", "palegreen"]
self._color_palette = color_palette.copy()
self._layout = layout = {
'title': 'Live Stream',
'xaxis': {
'title': "time"
},
'yaxis': {
'title': "values"
}
}
self._system_health_first_plot = True
console.debug(f'{target_system}.gemini')
if self._update_system_health_subscription is None:
self._update_system_health_subscription = aio.run(subscribe(f'{target_system}.gemini', self.update_system_health_plots))
async def update_system_health_plots(self, data, *args, **kwargs):
cpu, mem, partitions = data["cpu"], data["mem"], data["partitions"]
cpu_count = len(cpu)
partition_count = len(partitions)
format_str = "%Y-%m-%d %H:%M:%S"
if self._system_health_first_plot == True:
ts = time.time()
#t = datetime.datetime.utcfromtimestamp(t).strftime(format_str)
t = datetime.fromtimestamp(ts, tz=timezone.utc).strftime(format_str)
data_x, data_y, mem_x, mem_y, partition_x = [t]*cpu_count, cpu, [t], [mem], [t]*partition_count
# vvvv - usage per cpu core
columns = 2
rows = math.ceil(cpu_count / columns)
height = cpu_count * 200
self._ts_cpu._container_div.style.height = f"{height}px"
self._cpu_layout = cpu_layout = {"grid": {
'title': 'CPU usage',
'rows': rows,
'columns': columns,
'pattern': 'independent',
'roworder': 'bottom to top'}
}
color_palette = self._color_palette
traces: list = []
for x, y, cpu_nr, color, i in zip(data_x, data_y, range(cpu_count), color_palette, range(1, cpu_count+1)):
trace = {
'type': "scatter",
#'mode': "lines",
'name': f"core {cpu_nr}",
'line': {'color': color},
'fill': 'tozeroy',
'fillcolor': color,
'x': [x],
'y': [y],
'xaxis': f'x{i}',
'yaxis': f'y{i}'
}
traces.append(trace)
console.debug("traces:", traces)
self._ts_cpu.new_plot(traces, cpu_layout)
# ^^^^
# vvvv - Memory usage
mem_layout = {
'title': 'Memory Usage',
'xaxis': {
'title': "time"
},
'yaxis': {
'title': "Percent"
}
}
mem_traces = [{
'type': "scatter",
#'mode': "lines",
'name': "Memory usage",
'line': {'color': "red"},
'fill': 'tozeroy',
'fillcolor': 'red',
'x': mem_x,
'y': mem_y,
}]
self._ts_mem.new_plot(mem_traces, mem_layout)
# ^^^^
# vvvv - partitions
columns = 2
rows = math.ceil(partition_count / columns)
height = partition_count * 200
self._ts_partitions._container_div.style.height = f"{height}px"
self._partition_layout = partition_layout = {"grid": {
'title': 'Partition usage',
'rows': rows,
'columns': columns,
'pattern': 'independent',
'roworder': 'bottom to top'}
}
color_palette = self._color_palette
traces: list = []
for x, partition_mountpoint, partition_nr, color, i in zip(partition_x, partitions, range(partition_count), color_palette, range(1, partition_count+1)):
partition = partitions[partition_mountpoint]
y = partition["used"]
trace = {
'type': "scatter",
#'mode': "lines",
'name': partition_mountpoint,
'line': {'color': color},
'fill': 'tozeroy',
'fillcolor': color,
'x': [x],
'y': [y],
'xaxis': f'x{i}',
'yaxis': f'y{i}'
}
traces.append(trace)
console.debug("traces:", traces)
self._ts_partitions.new_plot(traces, partition_layout)
# ^^^^
self._system_health_first_plot = False
else:
t = time.time()
start_time = t - 60 * 5
end_time = t
start_time = datetime.fromtimestamp(start_time, tz=timezone.utc).strftime(format_str)
end_time = datetime.fromtimestamp(end_time, tz=timezone.utc).strftime(format_str)
t = datetime.fromtimestamp(t, tz=timezone.utc).strftime(format_str)
# vvvv- cpu usage
data = {"x": [[t]]*cpu_count, "y": [[c] for c in cpu]}
plot_indexes_to_modify = list(range(len(data["y"])))
cpu_layout = self._cpu_layout
for core in range(1, cpu_count+1):
if core == 1:
xaxis = "xaxis"
yaxis = "yaxis"
else:
xaxis = "xaxis" + str(core)
yaxis = "yaxis" + str(core)
cpu_layout[xaxis] = {
"type": 'date',
"range": [start_time, end_time],
'title': f'CPU Usage (core {core})',
}
cpu_layout[yaxis] = {
'title': "Percent",
"range": [0, 100]
}
await self._ts_cpu.relayout(cpu_layout)
self._ts_cpu.extend_traces(data, plot_indexes_to_modify)
# ^^^^
# vvvv - memory usage
mem_layout = {
"xaxis": {
"type": 'date',
"range": [start_time, end_time],
'title': 'Memory Usage',
},
'yaxis': {
'title': "Percent",
"range": [0, 100]
}
}
await self._ts_mem.relayout(mem_layout)
self._ts_mem.extend_traces({"x": [[t]], "y": [[mem]]}, [0])
# ^^^^
# vvvv- partition usage
data = {"x": [[t]]*partition_count, "y": [[p["used"]] for mp, p in partitions.items()]}
plot_indexes_to_modify = list(range(len(data["y"])))
partition_layout = self._partition_layout
for i, mountpoint in zip(range(1, partition_count+1), partitions):
partition = partitions[mountpoint]
if i == 1:
xaxis = "xaxis"
yaxis = "yaxis"
else:
xaxis = "xaxis" + str(i)
yaxis = "yaxis" + str(i)
partition_layout[xaxis] = {
"type": 'date',
"range": [start_time, end_time],
'title': mountpoint,
}
partition_layout[yaxis] = {
'title': "Used",
"range": [0, partition["total"]],
"tickformat": "bformat"
}
await self._ts_partitions.relayout(partition_layout)
self._ts_partitions.extend_traces(data, plot_indexes_to_modify)
# ^^^^
register_logic_class(Demo)
</ui-logic>
</ui-page>
</body>
</html>
The main changes are:
- all the calls to
.relayoutmethods are awaited update_system_health_plotsis made asynchronoussubscribeis rewritten (in Python instead of Javascript) as an asynchronous function, invoked throughaio.runininit_cpu_n_mem_plot- instead of
timer.set_interval, the infinite loop is created with the usualwhile True:block where the call toupdate_system_health_plotsis awaited, and the interval is performed byawait aio.sleep()
Hello @PierreQuentel Pierre, first of all thanks for your effort!
Apparently I did not explain my example code well enough. The JavaScript part is intentionally written in JavaScript and has replaced a JS library, which would have unnecessarily increased the complexity of the example. Initially I thought that this was necessary to cause the error.
I keep trying to learn more about the interpreter and solve errors myself, but unfortunately I don't have enough time at the moment to study it in detail.
The error on my part is due to the Plotly documentation not mentioning that a Promise object is returned here and I have not inspected this sufficiently.
Even if the Promise object is not awaited, I don't think there should be a RecurssionsError.
In a similar bug report, I have attached a build that contains a change I made that does not result in a ReccursionsError. I am still testing if there are no unwanted side effects.
You can find the customized version here: #2411 (comment)
I will upload the change in a separate branch later so that you can have a look at it.
Have a nice evening. 🙂
Hi @PierreQuentel, I have created a branch (fix_2390_and_2411) that contains a possible fix. Can you please check if I have overlooked something and if the change is in your interest?
Thanks a lot!
Yes, the change is ok and also fixes issue #2411, great ! Can you submit a Pull Request ?
Thanks again @moepnse !