Performance Question
mbriggs opened this issue · comments
Hi Le,
I have a question regarding perf. I am using flx through projectile on my projects files (~8k). When searching for something which contains many matches initially, flx will lock up for ~1s before it narrows to a point where it can display results.
What I am curious about is if I then backspace to nothing, and type in the same word, performance is smooth as silk. But if I dismiss flx and invoke it again, I see the same lock up. Is there anything I can do to cache whatever calculations flx is doing the first time through?
Even with that, flx is so much better then the alternatives I will live with it :)
Ok, digging into this a bit, it turns out that for large datasets, as you narrow down it is supposed to flip back to flx when it enters the threshold. Adding some debugging output to flx-ido-match shows me that it is getting passed the full set of candidates on each keystroke, so the threshold never gets entered.
I hacked up the way the choice was made here mbriggs@cc71f03 which now works for me. Would make a pull request, but I really didnt do it in a terribly elegant way.
So the idea is in these cases, you would decrease the threshold until reaching a compromise of acceptable performance and match happiness. Obviously it's not working as it should I'll mark it as a bug and investigate further later.
I see the behaviour you describe. Definitely a bug.
Actually I recall in 24.4 a bunch of ido caching is coming in, so the behaviour has changed a lot.
using GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.1.0, NS apple-appkit-1265.19) of 2014-04-01 on chillwind-yeti.local if that helps at all.
Can you try fix-ido-not-narrowing branch?
Amazing performance improvement! 👍
One thing is that these changes cause a conflict with ido-hacks, which probably isnt a problem, but you may want to document somewhere. Performance is absolutely fantastic overall :D
one thing is if you backspace, flx matching is not invoked until you type again (or if it is, the faces aren't showing up). REALLY not a big deal though
one thing is if you backspace, flx matching is not invoked until you type
again
Yeah I just realized this. I'll fix it soon (and ask you to test again).
On Wed, Apr 30, 2014 at 4:34 PM, Matt Briggs notifications@github.comwrote:
one thing is if you backspace, flx matching is not invoked until you type
again (or if it is, the faces aren't showing up). REALLY not a big deal
though—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-41846747
.
Le
What exactly is the conflict with ido-hacks?
On Wed, Apr 30, 2014 at 4:23 PM, Matt Briggs notifications@github.comwrote:
Amazing performance improvement! [image: 👍]
One thing is that these changes cause a conflict with ido-hacks, which
probably isnt a problem, but you may want to document somewhere.
Performance is absolutely fantastic overall :D—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-41845597
.
Le
if ido-hacks is there and flex matching is enabled, no candidates show up at all if you are above the flx threshold.
i think the main reason I still had ido-hacks was that it enabled ido in a few places that even ido-ubiquitous doesnt. Given that the main point of the mode is to speed up flex, and flex speed seems to not be an issue at all on 24.4/with this branch, I would honestly rather just extract the bits that I want out of it then bringing the full library in anyways.
Ok, the fix in this branched works with most cases, but I was able to create a perf issue (pretty sure it is a corner case).
https://gist.github.com/mbriggs/65ab7c201e0739dce80c
That is a find . of a project, where there is a HUGE amount of files in a few directories. When I specifically narrow to one of those mega-directories, followed by trying to narrow further, flx will hang for a moment in the transition between flex and flx
An example of that (going against the list in the gist) would be layoutmaasvai. I am on a 2.8 ghz i7, have gc-cons-threshold set to 20000000, and flx-ido-threshold set to 2000.
Yes, I think I understand the problem. It's the same as flx not activating
after backspacing.
On Wed, Apr 30, 2014 at 5:05 PM, Matt Briggs notifications@github.comwrote:
Ok, the fix in this branched works with most cases, but I was able to
create a perf issue (pretty sure it is a corner case).https://gist.github.com/mbriggs/65ab7c201e0739dce80c
That is a find . of a project, where there is a HUGE amount of files in a
few directories. When I specifically narrow to one of those
mega-directories, followed by trying to narrow further, flx will hang for a
moment in the transition between flex and flxAn example of that (going against the list in the gist) would be
layoutmaasvai. I am on a 2.8 ghz i7, have gc-cons-threshold set to
20000000, and flx-ido-threshold set to 2000.—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-41850561
.
Le
Can you give this a go?
I've fixed a typo in the threshold variable name.
The default has also been lowered from 6000 to 500.
I think there is a case now where flx can get the wrong list of candidates, will try to find a reproducable set of steps
Have you seen this wrong candidates bug again?
On Thu, May 1, 2014 at 10:01 AM, Matt Briggs notifications@github.comwrote:
I think there is a case now where flx can get the wrong list of
candidates, will try to find a reproducable set of steps—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-41912152
.
Le
I havent... i hit two situations where i thought i should have seen results but didnt just during normal work. it honestly just could have been user error >.>
That's fine.
I'll probably cut a new version in a few days and update the README to
recommend using flx as the "last mile" solution with a low
flx-ido-threshold.
On Fri, May 2, 2014 at 11:23 AM, Matt Briggs notifications@github.comwrote:
I havent... i hit two situations where i thought i should have seen
results but didnt just during normal work. it honestly just could have been
user error >.>—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-42043425
.
Le
nope, mostly sure it was user error :) been using https://github.com/topfunky/PeepOpen recently for finding project files though.
The fix has been merged into master