Closing error channel can panic
isaldana opened this issue · comments
If the callback returns an error (without having a fallback) an error is sent here: https://github.com/afex/hystrix-go/blob/master/hystrix/hystrix.go#L96.
If a timeout in timer.C occurs, an error will be sent here: https://github.com/afex/hystrix-go/blob/master/hystrix/hystrix.go#L125
Both errors can be triggered on the same call (i.e. if a timeout occurs and later the callback returns an error without a fallback function). If the error channel is closed after the caller receives the first error, the second error won't be able to be sent on a closed channel and panic. If this is expected behavior, should we document it here: https://github.com/afex/hystrix-go#waiting-for-output ? The workaround is not to close the channel and let it GC the second error.
Another side effect of this is that the fallback function is called twice.
where are you seeing the error channel being closed? channels should not be closed by receiving code, only by senders: http://golang.org/pkg/builtin/#close
you are right that there are some nuances around timeouts since go does not offer (afaik) a way to kill an in-progress goroutine. the behavior you identified is also seen if you have non-buffered response channel, and a command which times out but doesn't fail. your run function will eventually attempt to send to the channel, blocking indefinitely since you are likely no longer selecting on it. this causes goroutine leaks.
you do highlight a great point that we have nothing that will block the fallback from executing twice which is clearly an error. i'm going to use this issue to track that we need to change that behavior
- prevent post-timeout errors from triggering fallback