Unable to specify ThreadGroup with 'infinite iterations' in dsl
andy-tarr opened this issue · comments
When a threadgroup is specified as having loop count = 'inifinite the resulting jmx has
<intProp name="LoopController.loops">-1</intProp>
sample_jmx.zip
specifying this using the dsl:
threadGroup ("TG One") .rampTo(100, Duration.ofSeconds(3)) .holdIterating(-1)
results in the following error
Exception in thread "main" java.lang.IllegalArgumentException: Iterations must be >=0
at us.abstracta.jmeter.javadsl.core.threadgroups.DslDefaultThreadGroup.holdIterating(DslDefaultThreadGroup.java:251)
If the attached jmx is used with the latest jmx2dsl the output code has:
threadGroup() .rampTo(100, Duration.ofSeconds(3)) .holdIterating(-1)
but this won't work
just played around with this and realised that defining a TG like this results in the TG having inifinite loops
threadGroup ("TG One") .rampTo(100, Duration.ofSeconds(3))
so there are a couple of issues:
- dsl not intuitive regarding setting TG to have infinite loops
- jmx2dsl results in incorrect dsl code
Thank you @andy-tarr for reporting this, provide such detail and even a workaround!
How do you usually use such test plans? Do you only use them with the GUI and manually stop the test plan? I ask because in general having a testplan without a test execution limit (iterations or durations) sounds like not a good practice, and that's why I decided to add such check in the method (even though you circumvented with a workaround :)).
Hi @rabelenda, thanks for yor response. Yes obviously an infinitely running test is not necessarily desirable and we set the test duration externally, either in BlazeMeter or using Taraus yaml config.
So, taking into consideration that we are converting from JMX to DSL and something that could be desirable while using JMX (since you have GUI control, and might use other tools with the JMX like taurus or BZ to change it) may not be of much use while using the DSL (code based, and potentially BZ exec would be managed in same code with further code changes), and may even lead to undesired behaviors (eg: a process running without limits either locally or in CI/CD). Do you think that makes sense to support allowing to configure infinite iterations in thread group (without any limit in duration either)?
What do you think would be the best approach:
- Improve existing exception/error message to indicate that such config is not desirable due to potential issue of test plan running without limits, and advice on adding either an iterations or duration limit to the thread group.
- Log a warning, instead of throwing an exception when such condition is met. I prefer in general not to do this, since in general warnings tend to be ignored and just add noise to logs.
- Remove such exception, giving greater flexibility to users and allow them to potentially fall in such pitfalls (we usually in the API design try to help users and avoid common pitfalls, so this would go against such principle in some scenarios).
- Change conversion (not adding holdIterating method) to avoid failing execution only in scenario where you convert such cases even though would not throw any warning or error and you may end up with a test plan that runs indefinitely.
- Change conversion adding a default duration to thread group. This would avoid the thread group to run indefinitely, making the test plan "safer" to execute, but would also mean a change in semantics and may lead to "unexpected" change to users.
Any other ideas or comments?
I really appreciate any type of feedback.
@rabelenda If holdIterating(-1) is specified from jmx to dsl conversion then i think improving the existing exception would be sufficient. However, having a way to explicitly specify infinite TG iterations in the dsl is still desirable imo rather than it being a by product of not including holdIterating() in the TG definition.
Hello, thank you for the answer!
I am wondering still if makes sense to support specifying infinite iterations in the API.
The main use case I can think it may make sense to specify infinite iterations is when locally testing something and want just to execute until I stop the process. But that would be prone later on missing to change it and run indefinitely in a CI/CD pipeline. Would be preferable to force you, from my point of view, to define a maximum number of iterations or time and then if you want you can manually cancel it, but you are safe that the test plan will never run more than expected limit. I know JMeter GUI does not take this approach, but I think is due to the inherent UI interaction/semantics/behavior and thinking of CI/CD or pipeline execution just as a complementary/auxiliary/secondary functionality.
Another use case might be when you know before hand that you would run this in Blazemeter (or Octoperf), as a way of "ignoring" it since in any case you later on will specify the actual value for Blazemeter. But in such case, in jmeter DSL is just better to specify the limit you would expect in the thread group and let BlazemeterEngine use that (which is the default behavior when you don't specify settings in BlazemeterEngine).
Any other scenarios that you can come up and I might be missing?
We just released a new version which removes this restriction. I hope this solves the issue :).