Slow
atomkirk opened this issue · comments
What is causing it to be so slow? Is it exvcr itself or whatever vcr is using to intercept the network calls?
Network call happens only when recording the data (first time), but method mocking (stubbing methods) on each replaying involves certain overheads. There would be rooms for improvement, but haven't been able to identify good ways.
@parroty Would you mind elaborating a bit on where the speed problems ultimately reside? Is it in the mocking in meck itself? I'm asking, because it shouldn't be possible for one spec to interfere w/ another, and if you don't set clear_mock, they can. If it's possible to fix the speed, and make that the default behaviour, I think it'd be worthwhile. (I can try to help, but if you could direct me on what you alrady know about the problem, I'd appreciate it)
I'm afraid that I haven't been able to investigate in detail. Profiling or something may be one approach.
Hi,
I'm also analysing the performance of ExVCR, in my case a test with ExVCR is about 20x times slower than a test without, 230ms vs 20ms. The tests with the 3 adapters are equally slow so I think it's in the shared code. I've tried some tools to get some indicators where to look to fix the performance problem. With ExProf I got the following result on with this test: mix test test/adapter_hackney_test.exs:11. The two highest values in the uS / CALLS column are: :erlang.term_to_binary and :erlang.binary_to_term. It looks like it has something to do with the JSON Module.
Gist: https://gist.github.com/tompesman/7d15f60ccb3393cf929c608a5e081ea1
After some more research it looks like the mock.ex file is causing the delay. The Task.await(:infinity) method is waiting for :meck.expect(module_name, function, callback) to return. Not sure why there is a delay here.
Also really interested, some of my tests with only 2 requests take 500ms which should be almost instant when no database connections are being made.
Has anyway found a work-around?
I had the same issue with mocking, so instead mock for every cassette I implemented mocking on startup, see #158
With this workaround, an application that heavily uses cassettes now needs 120 seconds less to run mix test