awolden / brakes

Hystrix compliant Node.js Circuit Breaker Library

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reported request count never goes down?

johanhaleby opened this issue · comments

I'm using version 2.5.1 and I try to view the stream from the hystrix dashboard but it seems to me that the requestCount never goes down even though the no one calls the Brakes "command":

image

The data that is reported by the hystrix stream looks like this:

data: {"type":"HystrixCommand","name":"MyName","group":"MyGroup","currentTime":1487861651232,"isCircuitBreakerOpen":false,"errorPercentage":0,"errorCount":0,"requestCount":53,"rollingCountBadRequests":0,"rollingCountCollapsedRequests":0,"rollingCountExceptionsThrown":0,"rollingCountFailure":0,"rollingCountFallbackFailure":0,"rollingCountFallbackRejection":0,"rollingCountFallbackSuccess":0,"rollingCountResponsesFromCache":0,"rollingCountSemaphoreRejected":0,"rollingCountShortCircuited":0,"rollingCountSuccess":53,"rollingCountThreadPoolRejected":0,"rollingCountTimeout":0,"currentConcurrentExecutionCount":0,"latencyExecute_mean":758,"latencyExecute":{"0":600,"25":664,"50":724,"75":868,"90":906,"95":912,"99":951,"100":951,"99.5":951},"latencyTotal_mean":15,"latencyTotal":{"0":600,"25":664,"50":724,"75":868,"90":906,"95":912,"99":951,"100":951,"99.5":951},"propertyValue_circuitBreakerRequestVolumeThreshold":100,"propertyValue_circuitBreakerSleepWindowInMilliseconds":5000,"propertyValue_circuitBreakerErrorThresholdPercentage":0.5,"propertyValue_circuitBreakerForceOpen":false,"propertyValue_circuitBreakerForceClosed":false,"propertyValue_circuitBreakerEnabled":true,"propertyValue_executionIsolationStrategy":"THREAD","propertyValue_executionIsolationThreadTimeoutInMilliseconds":800,"propertyValue_executionIsolationThreadInterruptOnTimeout":true,"propertyValue_executionIsolationThreadPoolKeyOverride":null,"propertyValue_executionIsolationSemaphoreMaxConcurrentRequests":20,"propertyValue_fallbackIsolationSemaphoreMaxConcurrentRequests":10,"propertyValue_metricsRollingStatisticalWindowInMilliseconds":10000,"propertyValue_requestCacheEnabled":false,"propertyValue_requestLogEnabled":false,"reportingHosts":1}

My Brakes "command" looks like this (defined "globally" in the module):

const myHystrixCommand = new Brakes(queryBitBucket, {
    statInterval: 1000,
    threshold: 0.5,
    circuitDuration: 5000,
    timeout: 5000,
    name: 'MyName',
    group: 'MyGroup',
    isPromise: true
});

Am I doing something wrong or why is the requestCount always showing 53? (which is the total number of calls I've made).

Hey @johanhaleby, if you are using the default bucketSpan and bucketNum options, it will take
60 seconds for requests to stop being reported. Do the # of requests stay at 53 even after a minute of monitoring it?

@awolden Thanks! It does indeed go down when I wait 60 seconds as you imply. But why is the default options the way they are? Is bucketSpan the equivalent of metrics.rollingPercentile.bucketSize and bucketNum the equivalent of metrics.rollingPercentile.numBuckets? If so Hystrix defaults to 100 and 6?

The options don't exactly line up with the hystrix options. Brakes doesn't have a bucketSize option. It will store an indefinite amount of response times in a bucket.

The math that Brakes uses to handle buckets is a little different than hystrix, but essentially is the same. In hystrix, you define how long you want to store your rolling stats (default: 60s) and the number of buckets you want. It then divides those values to determine how long each bucket should live. In Brakes you say how long you want your bucket to live bucketSpan (default: 1000ms) and how many buckets you want bucketNum (default: 60) and multiplying those values together determines how long you store your rolling stats (default: 60s).

Hope that helps!

-Alex

Thanks a lot for the clarifications @awolden. And thanks for making the library available and supported, great work!