Giters

mesos / logstash

Logstash on Mesos

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Run executors without Docker

frankscholten opened this issue · comments

commented

@philwinder Please take a look at this

commented

This looks reasonable. Logstash is distributed as a zip, the executor is distributed as a jar. We can have a fileserver on the scheduler that provides the URI's so that Mesos can download them. Mesos is also able to extract from zips, so we can do that too. Then we just need the configuration options.

commented
  • Update Gradle build so it downloads logstash binary and packages it inside scheduler jar
  • Add cli options
  • Update code in TaskInfoFactory
  • Make logstash binary available from scheduler
  • Log command in TaskInfoFactory
  • Add Jar mode test in DiscoverySystemTest
  • Testing
  • Add --enable.docker flag to documentation
commented

Then we just need the configuration options

The executor should be picking them up automagically over protobuf. So what's the issue?

commented

@frankscholten you don't need a webserver. Spring aready has one. I had already done this. Also you need a Gradle task to download the logstash zip. See my code.

commented

@philwinder where is your code?

commented

@mwl I meant CLI options to enable jar mode. (Or whatever configuration type you use)

commented

@philwinder Ah! Just add field/getter/setter to org.apache.mesos.logstash.config.FrameworkConfig and Spring will do it's magic.

commented

It turns out unzip was not installed on the minimesos Docker images so the logstash zip was not unzipped.

Here are the Docker images used by minimesos https://hub.docker.com/r/containersol/mesos-agent https://hub.docker.com/r/containersol/mesos-master

Things to do

  • Update Gradle build so it downloads logstash binary and packages it inside scheduler jar
  • Add cli options
  • Update code in TaskInfoFactory
  • Log command in TaskInfoFactory
  • Add Jar mode test in DiscoverySystemTest
  • Test minimesos with updated images
  • Update logstash so it uses new minimesos version through Jitpack
  • Clean up
  • Fix DeploymentSystemTest.testDeploymentJar and check that logstash binary now runs correctly
  • Add --enable.docker flag to documentation
commented

A few open tasks left, which are not relevant anymore.