On Windows, gtm slows down commit, high cpu usage
ekbarber opened this issue · comments
When ever I commit, it takes close to a full minute for the commit command to complete, and when I look at my Task Manager I see that gtm.exe is using ~30% CPU. I'm attaching a screenshot of the TaskManager, as well as a screenshot of my system specs.
Any thoughts as to the cause of this, or is this just expected behavior?
What plugin are you using?
GitTimeMetric for SublimeText 3, downloaded via Package Control
It's odd that you are experiencing high CPU. GTM should only run occasionally and immediately exit.
Does it continuously run like this?
What else is running? I noticed that your CPU is at 60 percent? It also looks like most of the memory is in use at 83 percent. Am I correctly understanding the stats in the screenshot?
Please provide your gtm version? gtm --version
You are correctly understanding the stats. I do have other things running, so I certainly don't feel as though GTM is the sole cause for my cpu being generally high. However, I just found it odd that git commits now take 30 seconds to a minute after installing GTM. To be clear, GTM is not constantly running at high CPU, just when I go to commit or run gtm status. I should also note that gtm status takes somewhere around 10-15 seconds to complete as well.
According to gtm --version I'm running version 1.2.6.
I don't find this a huge issue by any means, and it's entirely possible its caused by a mix of things running on my PC. I just wanted to see if other people were experiencing something similar. Overall I'm loving GTM, definitely has helped me get a better sense of how much time I spend developing vs various admin tasks.
That definitely is slow and not acceptable performance. Normally these processes should take milliseconds to execute. Let's see if we can figure out the root cause.
When you run gtm status, gtm reads all the event files in the .gtm directory within the root of your project, calculates the time spent and then outputs the status.
Similarly, when you do a git commit it triggers the post-commit hook which in turn calls gtm commit -y. The commit process reads all the event files, calculates the time spent, creates a git note with the results and deletes the processed event files.
One thing we can try is creating a fresh git project, initialize gtm, edit some files and commit the changes after five minutes. Before you actually commit, check first the number of event files created in the .gtm directory. It would be ideal if you could provide the list of event files created. When making the commit note the time it takes to run.
Interestingly enough, when I create a new directory and init git and gtm on it, then run gtm status, the command completes almost instantly. So it must be something specific to the git repo I'm working with.
This repo does contain well over 10,000 files, and I did notice briefly yesterday that when I ran gtm status it appeared to be touching every file according to the Windows Resource Monitor. So could a large number of files cause subpar performance for the command?
That is good information. When running gtm status it doesn't look at the source files in the repo but it does first need to find the project's root directory. I use libgit2 to integrate with git for this. I wonder if there is something going on with that process.
What directory location are you in when running gtm status for the project? Are you in the root of the git repo or in a subdirectory?
You also may want to try running gtm clean and also gtm init on the repo.
I am running it from the root of the git repo. I'll try a gtm clean and report back
Sorry for the delay. I've done a gtm clean but no change. Any other suggestions? I've noticed gtm.exe spinning at ~10-20% cpu for 15-20 seconds after simply bringing the focus to SublimeText. So it seems like both gtm record and a gtm status are taking longer than expected.
I'm surprised it's both slow for recording and status. Recording is a very light process. It essentially writes small files in response to specific editor events. For instance, if you are in a single file editing it should only record at most an event every 30 seconds. You can test this by watching the .gtm directory within the root of your project. It would be interesting to know how many files it is actually writing and if your editor slows down when it writes files to that directory.
Another thing to test is how long the git hook take to run. You add timing code to the .git\hooks\post-commit. In Linux/OSX you can use the time command. I think there may be similar options for Windows.
Is possible that your computer is just resource starved in general?
It appears to be correctly writing the files, as I do see a new one pop up every 30 seconds at most. I just opened up SublimeText from a minimized (not closed) state, and then jumped over to the task manager to see this. The high disk usage is certainly interesting.
As far as my computer being resource starved, that's certainly a possibility, but I feel as though it's not the only issue here. Especially since when I went to set this up in a fresh repo the problem went away. I have to imagine it has something to do with the large number of files I have in the repo. You mentioned it needing to find the projects root directory, and using libgit2 for this. I feel as though that is a likely culprit, is there something I can do to help test that theory?
I've added an option to profile the gtm status command with the latest release. You can download it from here https://github.com/git-time-metric/gtm/releases.
From the command line run gtm status --profile. You should see output like this.
[⟳ 5m0s][⤸0]❯ gtm status --profile
2017/03/30 17:27:16 git.RootPath took 169.575µs
2017/03/30 17:27:16 git.RootPath took 52.32µs
2017/03/30 17:27:16 git.RootPath took 37.017µs
2017/03/30 17:27:16 metric.Process took 4.557237ms
2017/03/30 17:27:16 report.Status took 1.501809ms
5m 0s 100% [r] Terminal
5m 0s gtm
2017/03/30 17:27:16 status.Run took 11.138534ms
This will hopefully help identify the root cause of the performance issue.
Awesome, thanks for adding that! It metric.process is where the hangup is. Any thoughts as to why?
This is very helpful. Need a little more detail. Added some more profiling.
Please install the latest https://github.com/git-time-metric/gtm/releases/tag/1.2.8-beta-2 and re-run the gtm status --profile command. Thanks.
See attached
Those times are very odd. Almost everything is 0 seconds. Here's an example of what you should expect for timings.
profile.txt
Can you try running this on some of your other Git projects? Try ones that have good gtm performance.
I noticed in yours most of the times were in nanoseconds. Is it possible that the timing library being used isn't able to provide the same level of precision on Windows?
I created a brand new git repo and init gtm on it, created a test file, made a some edits over the course of a couple minutes, then ran gtm status --profile and below are the results
So on this one it's nice and fast, but I'm still seeing a lot of 0 readings.
I think you're right. Seems to be a difference in precision for the system clock or maybe a bug in Go on Windows.
This is interesting - from your test results with the slow git repo.
2017/04/03 08:31:20 scm.RootPath took 0s
2017/04/03 08:32:45 metric.buildCommitNote took 0s
2017/04/03 08:32:45 metric.Process took 1m25.7640302s
Notice the difference in timestamps between scm.RootPath and metric.buildCommitNote. metric.buildCommitNote is reporting 0s but I think it may be where all the time is spent based on the difference in timestamps.
That would actually make sense, as I mentioned the repos in which gtm is slow contain a very large javascript library that consists of thousands of files, all nested in hundreds of directories. So if the method to discover the RootPath has to traverse each directory, that would certainly explain the lag in performance.