Purpose of LOGSTASH_HEAP_SIZE
sadovnikov opened this issue · comments
What is the purpose of LOGSTASH_HEAP_SIZE
or --logstash.heapSize
?
- this parameter is being processed by the process (or container), which is already started. Therefore it does not influence resources, which are allocated to the scheduler process
- in the code
LOGSTASH_HEAP_SIZE
is being summed withEXECUTOR_HEAP_SIZE
as requirement for executor.
private boolean isNotEnoughRAM(ClusterState clusterState, Protos.Offer offer) {
return !hasEnoughOfResourceType(offer.getResourcesList(), "mem", executorConfig.getHeapSize() + logstashConfig.getHeapSize() + executorConfig.getOverheadMem());
}
Isn't it confusing?
Should we remove this parameter?
Setting --logstash.heapSize=H
means we set environment variable LS_HEAP_SIZE=Hm
passed to the Logstash process. This LS_HEAP_SIZE
is undocumented, but you can see here that it's used to set the -Xmx
in the JAVA_OPTS
for the Logstash process.
So --logstash.heapSize
is the maximum size, in megabytes, of the memory allocation pool of the Logstash process.
However, it's unclear to me why this should be a user-configurable option. We the developers should know what the heap size limit should be.
OK!! Now it actually makes sense!!
I got confused because there is also --executor.heap-size
and they used somehow together to determining resource requirements of the executor. And there are FIXME
comment in README.md.
Could you please update the description?
@sadovnikov Does #106 make it clearer?
@mwl I think so - I basically copied over the description above into the README
#106 does make it clearer
Thanks @sadovnikov. #106 has been merged already.