loadtesting and checking latency
okeuday opened this issue · comments
A goal is to determine the source of the 3 second max sample that appears in some of the loadtests and look at minimizing it. It appears to be a 2 second timer wheel delay with an extra 1 second when the process is on the wrong scheduler, not associated with the timer wheel, but that is just a guess.
Use dtrace, lock counter and system monitor (erlang:system_monitor(MonitorPid, [busy_port, busy_dist_port, {long_gc, 200}, {long_schedule, 200}, ...])., locking scheduler threads: erlang:system_monitor(MonitorPid, [{long_schedule, 2}, {long_gc, 25}, {large_heap, ...}]).).
{scheduler, N} is an undocumented option to spawn (not lock?) an Erlang pid to a scheduler thread, which may be helpful
Make sure to use erl command line options (+sbt db) +sbt tnnps and +sub true
External service bottleneck is being tracked at http://bugs.erlang.org/browse/ERL-140
With the changes in Erlang/OTP 21 (I/O Polling) and the addition of cloudi_service_request_rate this issue can be closed.