Start up error
nickhristov opened this issue · comments
Getting an error when trying to start up the service. Here is the error report:
=CRASH REPORT==== 21-Sep-2016::18:40:30 ===
crasher:
initial call: application_master:init/4
pid: <0.136.0>
registered_name: []
exception exit: {bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{function_clause,
[{lists,nth,
[1,[]],
[{file,"lists.erl"},{line,170}]},
{autocluster_util,node_name_parse,3,
[{file,"src/autocluster_util.erl"},
{line,222}]},
{autocluster_util,node_name,1,
[{file,"src/autocluster_util.erl"},
{line,173}]},
{autocluster_aws,
'-get_autoscaling_group_node_list/2-lc$^0/1-0-',
1,
[{file,"src/autocluster_aws.erl"},
{line,117}]},
{autocluster_aws,
get_autoscaling_group_node_list,2,
[{file,"src/autocluster_aws.erl"},
{line,117}]},
{autocluster,ensure_registered,2,
[{file,"src/autocluster.erl"},
{line,97}]},
{autocluster,init,0,
[{file,"src/autocluster.erl"},
{line,33}]},
{rabbit_boot_steps,
'-run_step/2-lc$^1/1-1-',1,
[{file,"src/rabbit_boot_steps.erl"},
{line,49}]}]}}}}
in function application_master:init/4 (application_master.erl, line 134)
ancestors: [<0.135.0>]
messages: [{'EXIT',<0.137.0>,normal}]
links: [<0.135.0>,<0.31.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1598
stack_size: 27
reductions: 98
neighbours:
Here is the configuration under /etc/rabbitmq/:
File /etc/rabbitmq/enabled_plugins:
[autocluster,rabbitmq_stomp].
File /etc/rabbitmq/rabbitmq.config:
[
{rabbit, [
{tcp_listeners, [5673]},
{log_levels, [{autocluster, debug}, {connection, info}]}
]},
{rabbitmq_stomp, [{tcp_listeners, [{"0.0.0.0", 61613} ]}]},
{rabbitmq_stomp, [{default_user, [{login, "<REDACTED>"},
{passcode, "<REDACTED>"}]}]},
{autocluster, [
{backend, aws},
{aws_autoscaling, true},
{aws_ec2_region, "us-west-2"},
{aws_access_key, "<REDACTED>"},
{aws_secret_key, "<REDACTED>"}
]}
].
Looks like a programming bug: attempting to access the nth element of an empty list ?
The plugin fails to parse your node's hostname.
This is not necessarily a bug. What does hostname -f output on the node?
There's no response and this is an issue that's specific to a particular hostname value => it should be closed unless we get a failing example.
Hi
I am having the same error on k8s ontop of aws.
Any idea what might be causing this?
Hostname command below:
root@rabbitmq-0:/# hostname -f
rabbitmq-0.rabbitmq-service.production.svc.cluster.local
root@rabbitmq-0:/# hostname -s
rabbitmq-0
Eniromental variables started with:
env:
# For consupmption by rabbitmq-env.conf
- name: MY_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: AUTOCLUSTER_LOG_LEVEL
value: "debug"
- name: RABBITMQ_USE_LONGNAME
value: "true"
- name: AUTOCLUSTER_TYPE
value: "k8s"
- name: AUTOCLUSTER_DELAY
value: "60"
- name: K8S_SERVICE_NAME
value: "rabbitmq-service"
- name: AUTOCLUSTER_CLEANUP
value: "true"
- name: AUTOCLUSTER_FAILURE
value: "stop"
- name: CLEANUP_WARN_ONLY
value: "false"
- name: K8S_HOSTNAME_SUFFIX
value: ".rabbitmq-service.production.svc.cluster.local"
- name: K8S_ADDRESS_TYPE
value: hostname
- name: LANG
value: "en_US.UTF-8"
- name: RABBITMQ_ERLANG_COOKIE
valueFrom:
secretKeyRef:
name: rabbitmq-prod-secret
key: erlang_cookie
- name: RABBITMQ_DEFAULT_USER
valueFrom:
secretKeyRef:
name: rabbitmq-prod-secret
key: username
- name: RABBITMQ_DEFAULT_PASS
valueFrom:
secretKeyRef:
name: rabbitmq-prod-secret
key: password
- name: K8S_TOKEN_PATH
value: /var/run/secrets/kubernetes.io/rabbitmq/token
- name: K8S_CERT_PATH
value: /var/run/secrets/kubernetes.io/rabbitmq/ca.crt
- name: K8S_NAMESPACE_PATH
value: /var/run/secrets/kubernetes.io/rabbitmq/namespace
Crash Report.
=CRASH REPORT==== 8-May-2017::14:53:38 ===
crasher:
initial call: application_master:init/4
pid: <0.127.0>
registered_name: []
exception exit: {bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{{badmatch,{error,enoent}},
[{autocluster_k8s,make_request,0,
[{file,"src/autocluster_k8s.erl"},
{line,59}]},
{autocluster_k8s,nodelist,0,
[{file,"src/autocluster_k8s.erl"},
{line,28}]},
{autocluster,ensure_registered,3,
[{file,"src/autocluster.erl"},{line,107}]},
{autocluster,init,0,
[{file,"src/autocluster.erl"},{line,33}]},
{rabbit_boot_steps,'-run_step/2-lc$^1/1-1-',1,
[{file,"src/rabbit_boot_steps.erl"},
{line,49}]},
{rabbit_boot_steps,run_step,2,
[{file,"src/rabbit_boot_steps.erl"},
{line,49}]},
{rabbit_boot_steps,
'-run_boot_steps/1-lc$^0/1-0-',1,
[{file,"src/rabbit_boot_steps.erl"},
{line,26}]},
{rabbit_boot_steps,run_boot_steps,1,
[{file,"src/rabbit_boot_steps.erl"},
{line,26}]}]}}}}
in function application_master:init/4 (application_master.erl, line 134)
ancestors: [<0.126.0>]
messages: [{'EXIT',<0.128.0>,normal}]
links: [<0.126.0>,<0.31.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1598
stack_size: 27
reductions: 98
neighbours:
=INFO REPORT==== 8-May-2017::14:53:38 ===
application: rabbit
exited: {bad_return,
{{rabbit,start,[normal,[]]},
{'EXIT',
{{badmatch,{error,enoent}},
[{autocluster_k8s,make_request,0,
[{file,"src/autocluster_k8s.erl"},{line,59}]},
{autocluster_k8s,nodelist,0,
[{file,"src/autocluster_k8s.erl"},{line,28}]},
{autocluster,ensure_registered,3,
[{file,"src/autocluster.erl"},{line,107}]},
{autocluster,init,0,
[{file,"src/autocluster.erl"},{line,33}]},
{rabbit_boot_steps,'-run_step/2-lc$^1/1-1-',1,
[{file,"src/rabbit_boot_steps.erl"},{line,49}]},
{rabbit_boot_steps,run_step,2,
[{file,"src/rabbit_boot_steps.erl"},{line,49}]},
{rabbit_boot_steps,'-run_boot_steps/1-lc$^0/1-0-',1,
[{file,"src/rabbit_boot_steps.erl"},{line,26}]},
{rabbit_boot_steps,run_boot_steps,1,
[{file,"src/rabbit_boot_steps.erl"},
{line,26}]}]}}}}
@wesselOC you are not getting the same error. A quick look at the code suggests that a read attempt on autocluster.k8s_token_path fails with an ENOENT (the file does not exist).
Therefore, according to your config, /var/run/secrets/kubernetes.io/rabbitmq/token does not exist at the time it is read.