High CPU usage during REPL debug for large input file
princespaghetti opened this issue · comments
Versions tested with: 0.11.1/0.12
While using fregot with a a custom set of rules and leveraging the regula library, the execution hangs while invoking the main deny. I let it run for 10 mins or so and it keep pretty high CPU usage the whole time.
Steps to recreate:
fregot repl --input tf-plan.json policy
:break data.rule.<rulename>.<etc>
data.main.deny
This seems to be due to a large tf-plan file (1.1 MB). Smaller files seem to work fine with the same library of rules.
@princespaghetti Thanks for reporting! That's interesting, but it's not entirely impossible to happen if you have accidentally quadratic code in a rule -- this happened to us a bunch of times as well. Most of the time the rule can be refactored to fix that. Were you able to run the rule in reasonable time using the vanilla Open Policy Agent?
Any chance you could get us some more information, e.g. which rule may be the culprit, or what the approximate amount of resources in the tf-plan file is?
Hey,
The rules run in a reasonable amount of time via opa, conftest, and via fregot when a breakpoint is not set. It's only on setting a breakpoint on a function within one of the rules that the situation arises. There's roughly 200 items in the resource changes. We also have a pretty decent sized set of rules so it may take some time to narrow down which rule is causing the problem
The rules run in a reasonable amount of time via opa, conftest, and via fregot when a breakpoint is not set
Okay, this indicates what the problem is.
When you're debugging, fregot disables all memoization / caching. The reason for that is otherwise you sometimes seem to jump around at random if results are already computed.
However, disabling these optimizations can cause huge regressions, since a lot of rules will need to be recomputed all the time. I'll see if we can add a way to leave them on during debugging, maybe with additional messages when caching causes a jump.