Load plugin emqx_auth_redis failed, cannot start plugin emqx_auth_redis
RodaBVB opened this issue · comments
Versions
- OS: Ubuntu 16.04.5 LTS
- Erlang/OTP: 18
- EMQ: 3.2.2
Topology
- 2x EMQX in cluster (4vCPU, 4GB RAM each)
- 3x RedisServer in sentinel (4vCPU, 4GB RAM each)
Description
Getting the following error while enabling "EMQ X Authentication/ACL with Redis" plugin from emqx_cli:
[Plugins] Load plugin emqx_auth_redis failed, cannot start plugin emqx_auth_redis for
{bad_return,
{{emqx_auth_redis_app,
start,
[normal,
[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,
emqx_auth_redis,
{shutdown,
{failed_to_start_child,
worker_sup,
{shutdown,
{failed_to_start_child,
{worker,
1},
{noproc,
{gen_server,
call,
[eredis_sentinel,
{get_master_req,
mymaster,
true}]}}}}}}}}}},
[{emqx_auth_redis_app,
start,
2,
[{file,
"/tmp/emqx/v3.2.2/_checkouts/emqx_auth_redis/src/emqx_auth_redis_app.erl"},
{line,
28}]},
{application_master,
start_it_old,
4,
[{file,
"application_master.erl"},
{line,
277}]}]
}
}}
}
load plugin error:
{emqx_auth_redis,
{bad_return,
{{emqx_auth_redis_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,emqx_auth_redis,
{shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
{noproc,
{gen_server,call,
[eredis_sentinel,
{get_master_req,mymaster,true}]}}}}}}}}}},
[{emqx_auth_redis_app,start,2,
[{file,
"/tmp/emqx/v3.2.2/_checkouts/emqx_auth_redis/src/emqx_auth_redis_app.erl"},
{line,28}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,277}]}]
}}
}
}
}
Here is my "emqx_auth_redis.conf" file:
auth.redis.type = sentinel
auth.redis.server = 192.168.x.x:26379,192.168.x.x:26379,192.168.x.x:26379
auth.redis.sentinel = mymaster
auth.redis.pool = 8
auth.redis.database = 3
auth.redis.acl_cmd = HGETALL mqtt_acl:%c
Firewall is open and connection to Redis (single node, or sentinel cluster ) from EMQX instances is working using redis-tool. Redis by using basic redis-cli command like "redis-cli -h 192.168.x.x: -p 26379"
EMQX is not able to connect to Redis sentinel, but is working when directed to single Redis instance.
Note:
Other applications use this Redis sentinel without issue. (nodejs app)
Do you have some directions on solving this issue?
Update:
EMQX logs: example of errors:
2020-01-20 15:10:24.705 [info] supervisor: {<0.2687.0>,ecpool_pool_sup}
started: [{pid,<0.2688.0>},
{id,pool},
{mfargs,
{ecpool_pool,start_link,
[emqx_auth_redis,
[{type,sentinel},
{pool_size,8},
{auto_reconnect,1},
{database,3},
{password,[]},
{sentinel,"mymaster"},
{servers,
[{"192.168.x.x",26379},
{"192.168.x.x",26379},
{"192.168.x.x",26379}]}]]}},
{restart_type,transient},
{shutdown,65535},
{child_type,worker}]
2020-01-20 15:10:24.707 [error] crasher:
initial call: eredis_sentinel:init/1
pid: <0.2691.0>
registered_name: []
exception error: no function clause matching
eredis_sentinel:read_sentinel({undefined,undefined}) (/tmp/emqx/v3.2.2/_checkouts/eredis/src/eredis_sentinel.erl, line 144)
in function eredis_sentinel:'-init/1-lc$^0/1-0-'/1 (/tmp/emqx/v3.2.2/_checkouts/eredis/src/eredis_sentinel.erl, line 93)
in call from eredis_sentinel:init/1 (/tmp/emqx/v3.2.2/_checkouts/eredis/src/eredis_sentinel.erl, line 93)
in call from gen_server:init_it/2 (gen_server.erl, line 374)
in call from gen_server:init_it/6 (gen_server.erl, line 342)
ancestors: [<0.2690.0>,<0.2689.0>,<0.2687.0>,emqx_auth_redis_sup,
<0.2685.0>]
message_queue_len: 0
messages: []
links: [<0.2690.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 987
stack_size: 27
reductions: 236
neighbours:
2020-01-20 15:10:24.707 [error] crasher:
initial call: eredis_client:init/1
pid: <0.2692.0>
registered_name: []
exception exit: {noproc,
{gen_server,call,
[eredis_sentinel,{get_master_req,mymaster,true}]}}
in function gen_server:call/2 (gen_server.erl, line 215)
in call from eredis_client:connect/1 (/tmp/emqx/v3.2.2/_checkouts/eredis/src/eredis_client.erl, line 321)
in call from eredis_client:init/1 (/tmp/emqx/v3.2.2/_checkouts/eredis/src/eredis_client.erl, line 89)
in call from gen_server:init_it/2 (gen_server.erl, line 374)
in call from gen_server:init_it/6 (gen_server.erl, line 342)
ancestors: [<0.2690.0>,<0.2689.0>,<0.2687.0>,emqx_auth_redis_sup,
<0.2685.0>]
message_queue_len: 0
messages: []
links: [<0.2690.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 987
stack_size: 27
reductions: 240
neighbours:
2020-01-20 15:10:24.708 [error] crasher:
initial call: ecpool_worker:init/1
pid: <0.2690.0>
registered_name: []
exception exit: {noproc,
{gen_server,call,
[eredis_sentinel,{get_master_req,mymaster,true}]}}
in function gen_server:init_it/6 (gen_server.erl, line 358)
ancestors: [<0.2689.0>,<0.2687.0>,emqx_auth_redis_sup,<0.2685.0>]
message_queue_len: 0
messages: []
links: [<0.2691.0>,<0.2692.0>,<0.2689.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 987
stack_size: 27
reductions: 366
neighbours:
2020-01-20 15:10:24.708 [error] supervisor: {<0.2689.0>,ecpool_worker_sup}
errorContext: start_error
reason: {noproc,{gen_server,call,
[eredis_sentinel,
{get_master_req,mymaster,true}]}}
offender: [{pid,undefined},
{id,{worker,1}},
{mfargs,
{ecpool_worker,start_link,
[emqx_auth_redis,1,emqx_auth_redis_cli,
[{type,sentinel},
{pool_size,8},
{auto_reconnect,1},
{database,3},
{password,[]},
{sentinel,"mymaster"},
{servers,
[{"192.168.x.x",26379},
{"192.168.x.x",26379},
{"192.168.x.x",26379}]}]]}},
{restart_type,transient},
{shutdown,5000},
{child_type,worker}]
2020-01-20 15:10:24.709 [error] supervisor: {<0.2687.0>,ecpool_pool_sup}
errorContext: start_error
reason: {shutdown,
{failed_to_start_child,
{worker,1},
{noproc,
{gen_server,call,
[eredis_sentinel,
{get_master_req,mymaster,true}]}}}}
offender: [{pid,undefined},
{id,worker_sup},
{mfargs,
{ecpool_worker_sup,start_link,
[emqx_auth_redis,emqx_auth_redis_cli,
[{type,sentinel},
{pool_size,8},
{auto_reconnect,1},
{database,3},
{password,[]},
{sentinel,"mymaster"},
{servers,
[{"192.168.x.x",26379},
{"192.168.x.x",26379},
{"192.168.x.x",26379}]}]]}},
{restart_type,transient},
{shutdown,infinity},
{child_type,supervisor}]
2020-01-20 15:10:24.710 [error] supervisor: {local,emqx_auth_redis_sup}
errorContext: start_error
reason: {shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
{noproc,
{gen_server,call,
[eredis_sentinel,
{get_master_req,mymaster,true}]}}}}}}
offender: [{pid,undefined},
{id,emqx_auth_redis},
{mfargs,
{ecpool,start_pool,
[emqx_auth_redis,emqx_auth_redis_cli,
[{type,sentinel},
{pool_size,8},
{auto_reconnect,1},
{database,3},
{password,[]},
{sentinel,"mymaster"},
{servers,
[{"192.168.x.x",26379},
{"192.168.x.x",26379},
{"192.168.x.x",26379}]}]]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,supervisor}]
2020-01-20 15:10:24.712 [error] crasher:
initial call: application_master:init/4
pid: <0.2684.0>
registered_name: []
exception exit: {bad_return,
{{emqx_auth_redis_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,emqx_auth_redis,
{shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
{noproc,
{gen_server,call,
[eredis_sentinel,
{get_master_req,mymaster,true}]}}}}}}}}}},
[{emqx_auth_redis_app,start,2,
[{file,
"/tmp/emqx/v3.2.2/_checkouts/emqx_auth_redis/src/emqx_auth_redis_app.erl"},
{line,28}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,277}]}]}}}}
in function application_master:init/4 (application_master.erl, line 138)
ancestors: [<0.2683.0>]
message_queue_len: 1
messages: [{'EXIT',<0.2685.0>,normal}]
links: [<0.2683.0>,<0.1253.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 610
stack_size: 27
reductions: 223
neighbours:
2020-01-20 15:10:24.713 [notice] application: emqx_auth_redis
exited: {bad_return,
{{emqx_auth_redis_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,emqx_auth_redis,
{shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
{noproc,
{gen_server,call,
[eredis_sentinel,
{get_master_req,mymaster,true}]}}}}}}}}}},
[{emqx_auth_redis_app,start,2,
[{file,
"/tmp/emqx/v3.2.2/_checkouts/emqx_auth_redis/src/emqx_auth_redis_app.erl"},
{line,28}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,277}]}]}}}}
type: temporary
2020-01-20 15:10:24.713 [error] [Plugins] Load plugin emqx_auth_redis failed, cannot start plugin emqx_auth_redis for {bad_return,
{{emqx_auth_redis_app,
start,
[normal,
[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,
emqx_auth_redis,
{shutdown,
{failed_to_start_child,
worker_sup,
{shutdown,
{failed_to_start_child,
{worker,
1},
{noproc,
{gen_server,
call,
[eredis_sentinel,
{get_master_req,
mymaster,
true}]}}}}}}}}}},
[{emqx_auth_redis_app,
start,
2,
[{file,
"/tmp/emqx/v3.2.2/_checkouts/emqx_auth_redis/src/emqx_auth_redis_app.erl"},
{line,
28}]},
{application_master,
start_it_old,
4,
[{file,
"application_master.erl"},
{line,
277}]}]
}}
}
}
Hi @RodaBVB ,
This is a duplicated issue to emqx/emqx#3219 , we will discuss it there.
Could you please close it here? Thank you.
Duplicate.