[EventCalendar] Memory leak when update events

[Aetf]

Coming from psifidotos/Latte-Dock#776, I finally find it's Event Calendar that leaking the memory.

Summary

I'm using latte dock with Event Calendar, and noticed that latte dock's memory usage constantly grows up, once to 4983MB after 3 days since startup.

After testing with multiple combinations of applets and layouts, I find Event Calendar is leaking memory whenever its making update.

Step to Reproduce

• Setup a google account for calendar sync
• Click on the refresh button to trigger an update
• Observe the memory usage increase of latte dock instance via system monitor

Actually I suspect it leaks memory when making network requests in general. During another 3 days' test, I didn't setup google account, so there shouldn't be any update automatically. But still the memory usage grows to 900M. I did have weather report setup though. So updating weather report also leaks memory.

I can reproduce the leak on v49 and v51.

[Chris Holland]

once to 4983MB after 3 days since startup.

What's your maximum RAM. How much is currently free? I do need to look into if

I've contemplated moving the google calendar HTTP request code into a separate python script, which will execute, then free up all memory, but it's a lot of work for something that already works.

Plus it might not be the QML HTTP requests leaking memory, but the giant JSON strings that aren't freed up which would still need to be parsed. I'm not sure if I need to manually delete things or if the Garbage Collector isn't firing (since you have extra RAM).

[Aetf]

I have 8G memory, and 2G swap on disk. Usually there's several GB free memory.

the giant JSON strings that aren't freed up which would still need to be parsed. I'm not sure if I need to manually delete things or if the Garbage Collector isn't firing (since you have extra RAM).

Then I suspect somewhere in the code still keeps reference to the JSON string. Because AFAIK, QML garbage collection has no knowledge about whole system memory usage, it simply frees object sooner or later after there's no reference to it.

Even though garbage collector is smart and take system memory usage into acccount, I've seen cases that I have several other large applications open, and the memory is completely used up, and the applet is still using over 4G memory.

[Aetf]

Also, are those giant JSON strings tens of MB large? Because each time I manually trigger the update, the memory usage increases by 10-20 MB.

[Chris Holland]

Commenting out the agendaListView.delegate (link), which draws the day + events in the agenda, didn't increase the memory usage. So it's not the HTTP stuff.

Something funky is going on in AgendaListItem.qml. I load several inputs (new event form), but are loaded dynamically...

Hmmmmm!

import "../code/WeatherApi.js" as WeatherApi

I betcha it's that bit there. It's probably loading the entire WeatherApi.js and everything it imports for every single agenda day. I will try testing after supper.

[Chris Holland]

I don't want you to think I consider this low priority. I've spent several programming sessions trying to isolate the leak. I've basically boiled it down to I'm doing something wrong. One of my property bindings is apparently using 5Mb... I think. Qt doesn't clean it up at least, maybe because it's referencing something that doesn't get destroyed when the "agenda day item" is destroyed on updates/month change.

I've tried valgrind --tool=massif plasmoidviewer to at least visualize Qt's C++ memory allocation.

There appears to be a lot of small things that aren't grouped that is using more and more memory, but the red section at the bottom has something funky going on with the scriptValueForContext.

[Aetf]

Thanks for the effort! I know it's really hard to deal with memory issues.

The graph generated by valgrind looks really helpful, but I haven't used it successfully personally in my project... Anyway, have you tried to get a breakdown in valgrind after the memory usage goes up, say ~1GB? Then it may be easier to spot which portion to takes most significant part of the heap, especially when comparing to the breakdown of normal memory usage (~100MB).

[Chris Holland]

v53 has a few related improvements, but the leak is still there.

• Refactored code to draw all event on loading them instead of a slight lag when scrolling when it tries to generate them on demand.
• When fetching events, it will now wait a little bit for another calendar before redrawing everything (this would create and destroy pointless AgendaListItems).
• Reuse existing event placeholders instead of deleting them all and generating new ones. This alleviates some of the memory creep (but not all).

---

You can ignore everything below.

I've managed to get around to compiling plasma-sdk/plasmoidviewer by itself inside Qt Creator. Achieving this means I can now use it's QML profiler.

I first modified the plasmoidviewer/CMakeLists.txt like so.

Then I added a few build steps.

1. kmake is my personal script to cmake then make.
2. Then I copied the package folder from the eventcalendar repo to plasmoidviewer/package/... This allows Qt Creator to "add the files to the project". If it's not a child directory of the source directory so that Qt Creator plays nice.
   
3. Then click the "Debugger" dropdown and select "QML Profiler"
   
4. Click "Start"
   
5. Click "Stop" after you finish playing with it. Do not close the window, as it doesn't "stop" the profiling.
   

Now I can see cool stuff like:

Timeline (as of v53)

Which is really useful for visualizing the call tree, and the what's taking so long.

From left to right.

1. "Today" button clicked (refreshes events)
2. The locally generate debugging events are finished being generated. This is pretty quick. We now wait a little bit to see if another calendar is ready within 400ms before we bother updating the UI.
3. Okay, no other calendars were ready, so lets draw an update. This requires a large chunk of time do to.
4. Ah, looks like the Google Calendar events we fetched are ready. Parsing them is fairly quick to do, but lets wait for if anything else asks us to redraw the UI (like clicking next month while it's still loading).
5. Okay, nothing else happened so we can redraw again.

Before v53, it wasn't "waiting" long enough, and it would essentially redraw the UI 4 times (I have 1 debug calendar and 3 google calendars). Which meant the UI was blocked for 2-4 seconds during updates.

To compare, here's a timeline from the v52 tag.

Other cool things:

Flame Graph (as of v53)

Statistics (as of v53)

[Chris Holland]

KDE Neon recently updated to Qt 5.11 / Plasma 5.13 / Frameworks v5.47, and plasmashell seems to be sticking around 200 Mb max.

[Aetf]

Yes, I can confirm that the leaking is not present with the latest update. Thanks for the investigation!