How to debug resource intensive process

[Cosmin Stamate]

Hi! I'm trying to integrate import-js in a fairly large Ember project and I'm encountering an issue that I don't know how to debug.

Initially I tried using the Sublime plugin, but I dropped to the CLI so that the Sublime plugin won't interfere in any way with what I'm seeing.

Basically I tried running importjs fix "path/to/file" and the process just hangs, consuming quite some resources. After ~10 minutes it got to 2 cores on 100% and ~10% RAM (out of 16GB), which I suppose shouldn't happen.

I tried the debug logging level setting, but there was no output at all after running the command. The log file doesn't seem to have anything relevant either.

This is my configuration:

module.exports = {
    excludes: [
        './config/**',
        './dist/**',
        './lib/**'
        './ops/**',
        './public/**',
        './vendor/**',
        './node_modules/**',
        './app/**'
    ],
    logLevel: 'debug',
    tab: '    ',
    groupImports: false,
    sortImports: false,
};

Is there anything obvious I'm missing, or is there any other way to make the command more verbose, so that I can see where it's hanging?

[Frederick Stark]

importjs writes logs to a temporary directory that you can find by using importjs logpath
So to watch the logs it's useful to tail it:

tail -f $(importjs logpath)

If you need more detail in the logs, you can debug by cloning this repo to your computer and using npm link to use the local version instead of the packaged one.
Then add your own log statements to figure out what it's doing

[Cosmin Stamate]

I figured it out. I had 3 pretty big folders that I forgot to exclude, and the ModuleFinder and the Watcher had a pretty hard time processing all of them.

In case anyone else stumbles upon this, the culprits were tmp, _gsdata_ and tests/snapshots. The first one is especially relevant to Ember projects, since all ember-cli projects currently have that folder, and it's pretty big. The second one is particular to my local setup, and the third one is particular to our project.

After excluding these, it still takes a couple of seconds for the first command to finish, because of the size of the project, but the subsequent commands are pretty fast.

[Henric Trotzig]

Glad you figured this out! I wonder if there's anything we can do to prevent others from running into the same issue? Similar to how tools like Command-T has a CommandTMaxFiles config, we could have something similar in .importjs.js, e.g. maxFiles. If this number is reached, we'd log something and stop indexing.

[Cosmin Stamate]

Yeah, that sounds good. It should be investigated a bit further if only the number of files itself was responsible, as that ember-cli tmp folder contains various files and links, and maybe there was something else blocking the tool. But a maxFiles limit would definitely be a good addition, I think.