Logstash Mesos Authentication
akamalov opened this issue · comments
Does Logstash currently support framework authentication? If it odes, will that be MESOS_PRINCIPAL, MESOS_SECRET variables ?
Tried to deploy it with the following Marathon JSON file:
{
"id": "/logstash",
"cpus": 1,
"mem": 1024.0,
"instances": 1,
"container": {
"type": "DOCKER",
"docker": {
"image": "mesos/logstash-scheduler:0.10-RC1",
"network": "HOST"
}
},
"env": {
"MESOS_ZK": "leader.mesos:2181",
"MESOS_ZOOKEEPER_SERVER": "leader.mesos:2181",
"MESOS_MASTER": "leader.mesos:5050",
"MESOS_PRINCIPAL": "elasticsearch",
"MESOS_SECRET": "dc5200a2a22b944d202e",
"FRAMEWORK_NAME": "logstash",
"MESOS_ROLE": "logstash",
"MESOS_USER": "root",
"LOGSTASH_HEAP_SIZE": "64",
"LOGSTASH_ELASTICSEARCH_URL": "http://node264.local.net:31100",
"EXECUTOR_CPUS": "0.5",
"EXECUTOR_HEAP_SIZE": "128",
"ENABLE_COLLECTD": "true",
"ENABLE_SYSLOG": "true",
"ENABLE_FILE": "true",
"ENABLE_DOCKER": "true",
"EXECUTOR_FILE_PATH": "/var/log/* "
}
}
Error output from a docker logs:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/logstash-mesos-scheduler.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/logstash-mesos-scheduler.jar!/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
|\ /|
| \ / |
| / \ |
|/ \|
/ \ /|
/ \ / | .____ __ .__
\ | | | | ____ ____ _______/ |______ _____| |__
\ | | | | / _ \ / ___\/ ___/\ __\__ \ / ___/ | \
| | / | |__( <_> ) /_/ >___ \ | | / __ \_\___ \| Y \
| |/ |_______ \____/\___ /____ > |__| (____ /____ >___| /
| / \/ /_____/ \/ \/ \/ \/
|/ :: Running Spring Boot 0.1.0 ::
2016-03-22 17:50:14.029 INFO 1 --- [ main] o.a.m.logstash.scheduler.Application : Starting Application v0.1.0 on node263.local.net with PID 1 (/tmp/logstash-mesos-scheduler.jar started by root in /)
2016-03-22 17:50:14.053 INFO 1 --- [ main] o.a.m.logstash.scheduler.Application : No active profile set, falling back to default profiles: default
2016-03-22 17:50:22,966:1(0x7f840fea8700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-03-22 17:50:22,966:1(0x7f840fea8700):ZOO_INFO@log_env@716: Client environment:host.name=node263.local.net
2016-03-22 17:50:22,966:1(0x7f840fea8700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-03-22 17:50:22,966:1(0x7f840fea8700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-327.4.4.el7.x86_64
2016-03-22 17:50:22,966:1(0x7f840fea8700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Thu Dec 17 15:51:24 EST 2015
2016-03-22 17:50:22,967:1(0x7f840fea8700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2016-03-22 17:50:22,967:1(0x7f840fea8700):ZOO_INFO@log_env@741: Client environment:user.home=/root
2016-03-22 17:50:22,967:1(0x7f840fea8700):ZOO_INFO@log_env@753: Client environment:user.dir=/
2016-03-22 17:50:22,967:1(0x7f840fea8700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=leader.mesos:2181 sessionTimeout=1000 watcher=0x7f846580d600 sessionId=0 sessionPasswd=<null> context=0x7f8408001160 flags=0
2016-03-22 17:50:22,974:1(0x7f840eca0700):ZOO_INFO@check_events@1703: initiated connection to server [192.168.158.237:2181]
2016-03-22 17:50:22,979:1(0x7f840eca0700):ZOO_INFO@check_events@1750: session establishment complete on server [192.168.158.237:2181], sessionId=0x2531f1acaa40041, negotiated timeout=4000
I0322 17:50:23.387738 22 sched.cpp:164] Version: 0.25.0
2016-03-22 17:50:23.403 INFO 1 --- [ main] o.a.m.logstash.scheduler.Application : Started Application in 11.432 seconds (JVM running for 12.583)
I0322 17:50:23.409267 37 sched.cpp:262] New master detected at master@192.168.158.237:5050
I0322 17:50:23.410462 37 sched.cpp:272] No credentials provided. Attempting to register without authentication
I0322 17:50:23.414841 37 sched.cpp:1024] Got error 'Framework at scheduler-e067cb07-f371-4645-8366-7faeef756b19@192.168.158.97:57528 is not authenticated'
I0322 17:50:23.414927 37 sched.cpp:1805] Asked to abort the driver
I0322 17:50:23.416898 37 sched.cpp:1070] Aborting framework ''
Unfortunately Mesos authentication isn't supported before we upgrade to Mesos-starter 0.2, which is due very soon.
Dang... :(
@mwl Hey Martin, wondering if Mesos-starter 0.2 upgrade has happened and the current release of logstash does support Mesos authentication, by chance?
Thanks again!!
Alex