karatelabs / karate

I am currently attempting to do some performance testing using gatling against an application, and some of the mock servers have a lot of different Scenarios to match against (more than 50, less than 100).

I'm finding that that MockHandler.isMatchingScenario is using more processing than the actual application logic I'm trying to performance test. Sometimes the responses coming back from MockServer will take over 1000 ms when there is no delay configured.

As best as I can tell the vast majority of slowness is coming from JS.eval, which leads me to believe that I should perhaps look to use a different MockServer solution, but I thought I'd see if this is an expected outcome for the way MockServer is designed?

I did attempt to put the more likely to match scenarios at the top, which improved the situation dramatically, but it did not solve the problem of inconsistent response times. At that point performance was still hindered by things like initEngine which seems to occur for every incoming request into the mock server.

I realize this probably isn't a bug report you can use at this time, but I thought it might be worth letting you know.

@sturose thanks for the report and I will keep it open. it is quite likely we can use code from #2530 to replicate

besides JS eval, there is a global lock for every request which in future we need to make user-opt-out via a tag:

karate/karate-core/src/main/java/com/intuit/karate/core/MockHandler.java

Line 153 in 30e93ed

public synchronized Response handle(Request req) { // note the [synchronized]

I'm pretty sure the work we are doing to replace the JS engine will help in case it is the JS eval which is the bottleneck. refer: #2546

finally for completeness - there is this "alternate" way to implement mocks, but my guess is you may want to stick to the simpler way: https://github.com/karatelabs/karate/wiki/Karate-JavaScript-Mocks

MockServer/MockHandler performance