api7 / lua-resty-etcd

Nonblocking Lua etcd driver library for OpenResty

Home Page:https://api7.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

bug: watch responses may be incorrectly parsed in the case of ETCD being proxied by a load balancer like nginx

nic-6443 opened this issue · comments

commented

We have encountered some problems in using apisix, because our etcd uses nginx as a proxy. When etcd returns event responses as http chunk after passing through nginx proxy, the following two situations may occur:

  1. the data of an event is larger than the proxy_buffer_size of nginx, the origin chunk will be split into two chunks by nginx and sent to apisix, resulting in the body_reader() getting an incomplete json, and then the parsing will fail.
  2. two consecutive events are merged into one chunk by nginx and sent to the client, which will also cause apisix to fail to parse the json.

local function read_watch()
while(1) do
body, err = res.body_reader()
if not body then
return nil, err
end
if not utils.is_empty_str(body) then
break
end
end
body, err = decode_json(body)

I think the root cause of the above problem is that apisix assumes that each chunk in the watch response is a complete event, in fact, a chunk may be a partial event or multiple events, so we should take these into account.

Since the raw data returned by etcd carries a newline at the end of each complete event, I think we can use newlines as a event splitter to solve the above problem.