Performance issues
brendt opened this issue · comments
Visiting the RFC detail page, while logged in with a user (maybe only admin users, need to verify), leads to a memory overflow error on the production server. The same problem doesn't happen locally :/
Need to further investigate.
This is why I don't like Livewire, it's impossible to debug when an error happens. It just says that memory overflow, even though we use around 100 MB on the page
The only way you can try to debug, is to get the dump of the database, and import everything locally. There is a big chance that you will be able to reproduce it.
I did copy the database, but it didn't happen locally :(
Hey @brendt @SerhiiCho , I tried to debug this locally but not sure 100% sure I understand what is happening here.
Here is what I did, I changed the seeder class to create a lot of arguments and votes, And I have this monster
I know that is not the same as the real db, but I think we are not really interested in the exact db but rather the amount of data.
Here are my findings so far:
- Locally in dev mode the detail page of rfc is quite slow with this amount of data:
- 9+ seconds as logged in as admin
- 4 + seconds as normal user
- 1.2 seconds as guest
so it's clear that we are doing something extra for logged in user and especially for admins
- I tried profiling the app with xdebug:
- This is sorted by memory:
- This is sorted by calls:
- This is sorted by own time:
- This is sorted by memory:
- I noticed that we are loading quite a lot of models especially for logged in users:
- Logged in as admin:
- Logged in as guest:
- Logged in as non admin:
- Logged in as admin:
Not 100% sure where the problem is, tried to comment out some templates indeed I saw some improvement in load time and some minor improvement in terms of models loaded.
I'm wondering, if this lines here would increase the load of the server by a lot if the currently authenticated user has quite a lot of arguments and viewedArguments, (I'll try to test this).
Wow @vsergiu93! You did quite a lot of debugging 👍 I thought the pagination will prevent all of the memory issues, but it seems like this page needs a good refactoring.
I would gladly optimize it if it wasn't Livewire. I felt the slowness of this page a long time ago, but I thought it will improve over time.
I used to writing API endpoint in PHP and using something like Vue on the frontend. Writing JavaScript with PHP looks odd to me, but I understand it's easier for backend developers.
@SerhiiCho I'm also not to that convinced about Livewire :), I mean is nice and all but it seems to be a bit confusing to work with, I'm also used to write apis for backends and use front-end frameworks like vue or react to consume them, but at the same time I do see some benefits with using Livewire especially if you hate javascript, or if you are a one man team and don't want to complicate yourself with another codebase and managing it and so on.
But what is clear for me is that we load way to many models to display just 15 items per page, on the rfc details page, I think there is room for improvements there.
@brendt can you tell me what are the settings/limits of the prod php.ini? maybe we should try to have the same settings locally, like what is the memory_limit not sure if you already shared this info somewhere else.
We should also consider querying with SELECT. So that we don't load columns that we don't use
Would it be an idea to gradually refactor out livewire? Starting with this page? I'm not sure about the implications though, since there's quite a lot going on, and my Vue knowledge is very rusty :/
About the memory limit: I'd need to check when I'm at my desk. But I'm pretty sure we've hit the root of the problem: we're loading in way too much data, especially when admin users are logged in. Thanks a lot for the debugging @vsergiu93 , awesome work!
I've update the title of this issue to better reflect the problem
By the way, Livewire might not be the root cause of the issue. We're loading quite a lot of model relations in memory, so that we don't need to perform any additional queries. For example: every argument loads each vote, and for that vote again the argument, so that we have easy access to data without having to worry about additional queries. Of course, at the cost of memory.
As an example, one of the features we need this for, is to check whether a user has voted for an argument. The better approach here is to have a separate database table to simply lists per user which arguments they voted for; it's a simple user_id argument_id table, which can be indexed and is super fast + low on memory.
I believe these kinds of optimisations will help us solve the root problem.
Hey @brendt, just to be clear I'm not suggesting to move away from Livewire, I was just expressing that I'm not sure if I like it or not, but that is just me.
I think the whole refc detail page needs to be refactored, but indeed I would start with lowering as much as possible the amount of eloquent models loaded.
Also, if you give some of the relevant php.ini values, like memory_limit I can try to have the same locally.
@vsergiu93 it's 512M, but that's the same limit I have locally, and I don't encounter any problems there :/
@brendt, Hmm this is weird, in any case I'll try to put this limit on my end as well, but It will be afternoon, in any case we kind of know now what the problem is. I just want to have going forward the more or less the same environment as in prod.
I did some more testing and made some optimizations: 6cd8d20
I temporarily disabled argument views, since they are very poorly implemented at the moment.
However, even with these changes, production is still crashing, only when logged in as an admin user. It's super annoying that I can't seem to reproduce the problem locally: I have the production database, am logged in as the admin user, and experience no problems. The debug bar also doesn't show an excessive amount of models in memory or queries.
Just writing it down here in case someone's debugging as well
It might be the new two column layout uses too much memory. What if we unroll changes before that pull request #404 and check if the memory issue disappears? Just to checkout into old commit and see if it works. If it is, I will undo the two column layout changes in a new PR
Ok good idea :) do you need any help with that?
It's done! I undo all the 2 column layout changes in this pull request. I already merged it
We'll see if it works on production
Unfortunately the problem's still there. I'll try to take a closer look at it this week :)
Did some more testing. The two biggest issues are these:
- Related RFCs at the bottom of the page — this can probably be fixed easily
- Unviewed arguments — this is more difficult to solve and will need a proper refactor
I've disabled both for now:
I think I've just resolved the issue in PR #457.
I believe this fixes the memory issue on the arguments page. I've noticed that options.blade.php blade component was causing memory issues for some reason. After this PR, memory usage dropped in 5 times if the debug bar is not lying. I hope it will solve the issue on production server as well.
I also enabled the hasSeenArgument() method on the User model since I haven't noticed any performance issues related to this particular method. We'll see, if anything, we can just disable or rewrite this method
Well it's definitely much faster now! Within acceptable range for admin users 🥳