Giters

upmc-enterprises / elasticsearch-operator

manages elasticsearch clusters

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Operator Panic - runtime error: slice bounds out of range

wwojcik opened this issue · comments

commented

Version: 0.3.0
kube version: 1.11.7
error:

kubectl -n monitoring logs -f elasticsearch-operator-845f78c79f-m5nc8                                                                                                                                                        (next.brickvest.com/dev)
time="2019-03-07T10:43:43Z" level=info msg="elasticsearch operator starting up!"
time="2019-03-07T10:43:43Z" level=info msg="Using Variables:"
time="2019-03-07T10:43:43Z" level=info msg="   enableInitDaemonset: true"
time="2019-03-07T10:43:43Z" level=info msg="   baseImage: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0"
time="2019-03-07T10:43:43Z" level=info msg="Using InCluster k8s config"
time="2019-03-07T10:43:43Z" level=info msg="SKIPPING: already exists \"elasticsearchclusters.enterprises.upmc.com\""
time="2019-03-07T10:43:43Z" level=info msg="Daemonset default/elasticsearch-operator-sysctl already exist, skipping creation ..."
time="2019-03-07T10:43:43Z" level=info msg="Found cluster: logging"
time="2019-03-07T10:43:43Z" level=info msg="use-ssl false"
time="2019-03-07T10:43:43Z" level=info msg="Found 1 existing clusters "
time="2019-03-07T10:43:43Z" level=info msg="Watching for elasticsearch events..."
time="2019-03-07T10:43:44Z" level=info msg="Process Elasticsearch Event ADDED"
time="2019-03-07T10:43:44Z" level=info msg="--------> Received ElasticSearch Event!"
time="2019-03-07T10:43:44Z" level=info msg="-----> Stop scheduler logging-monitoring"
time="2019-03-07T10:43:44Z" level=info msg="Found cluster: logging"
time="2019-03-07T10:43:44Z" level=info msg="use-ssl false"
time="2019-03-07T10:43:44Z" level=info msg="Using [upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_1] as image for es cluster"
time="2019-03-07T10:43:44Z" level=info msg="use-ssl false"
time="2019-03-07T10:43:45Z" level=info msg="CronJob elastic-logging-create-repository succesfully created ! "
time="2019-03-07T10:43:45Z" level=info msg="CronJob elastic-logging-snapshot succesfully created ! "
time="2019-03-07T10:43:45Z" level=info msg="--------> ElasticSearch Event finished!"
panic: runtime error: slice bounds out of range

goroutine 24 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processMasterPodEvent(0xc42034ca40, 0xc420762000, 0x10930e4, 0x4)
	/home/travis/gopath/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:294 +0x652
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processPodEvent(0xc42034ca40, 0xc420762000, 0x0, 0x0)
	/home/travis/gopath/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:277 +0x101
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents.func1(0xc4201a6000, 0xc42034ca40, 0xc42033ec60, 0xc4203a21e0, 0xc420362710)
	/home/travis/gopath/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:109 +0x1f8
created by github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents
	/home/travis/gopath/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:105 +0x78

cluster config:

apiVersion: enterprises.upmc.com/v1
kind: ElasticsearchCluster
metadata:
  name: logging
  namespace: monitoring
spec:
  kibana:
    image: docker.elastic.co/kibana/kibana-oss:6.1.3
  cerebro:
    image: upmcenterprises/cerebro:0.7.2
  elastic-search-image: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_1
  client-node-replicas: 1
  master-node-replicas: 1
  data-node-replicas: 1
  network-host: 0.0.0.0
  use-ssl: False
  zones: []
  data-volume-size: 10Gi
  java-options: "-Xms512m -Xmx512m"
  snapshot:
    scheduler-enabled: true
    bucket-name: bvn-es-logging-snapshots
    cron-schedule: "@every 30m"
    image: upmcenterprises/elasticsearch-cron:0.0.4
  resources:
    requests:
      memory: 512Mi
      cpu: 500m
    limits:
      memory: 1024Mi
      cpu: '1'
  storage:
    type: standard
    storage-class-version: volume.alpha.kubernetes.io/storage-class
    volume-reclaim-policy: Delete
commented

Same error here but on a different step. My cloud is Azure, could be something related with S3 snapshot functionality?

time="2019-03-08T19:07:08Z" level=info msg="elasticsearch operator starting up!"
time="2019-03-08T19:07:08Z" level=info msg="Using Variables:"
time="2019-03-08T19:07:08Z" level=info msg="   enableInitDaemonset: true"
time="2019-03-08T19:07:08Z" level=info msg="   baseImage: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0"
time="2019-03-08T19:07:08Z" level=info msg="Using InCluster k8s config"
time="2019-03-08T19:07:08Z" level=info msg="SKIPPING: already exists \"elasticsearchclusters.enterprises.upmc.com\""
time="2019-03-08T19:07:09Z" level=info msg="Daemonset default/elasticsearch-operator-sysctl already exist, skipping creation ..."
time="2019-03-08T19:07:09Z" level=info msg="Found 0 existing clusters "
time="2019-03-08T19:07:09Z" level=info msg="Watching for elasticsearch events..."
panic: runtime error: slice bounds out of range

goroutine 40 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processDataPodEvent(0xc0003097a0, 0xc0004fece0, 0x10b0148, 0x4)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:285 +0x205
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processPodEvent(0xc0003097a0, 0xc0004fece0, 0x0, 0x0)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:275 +0x137
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents.func1(0xc00016e060, 0xc0003097a0, 0xc0002ec2a0, 0xc00016ede0, 0xc0003666f0)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:109 +0x1ed
created by github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:105 +0x78```
commented

Also getting this error.

time="2019-03-11T21:11:50Z" level=info msg="use-ssl false"
time="2019-03-11T21:11:50Z" level=info msg="--------> ElasticSearch Event finished!"
panic: runtime error: slice bounds out of range

goroutine 54 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processDataPodEvent(0xc0002fc140, 0xc0004eb350, 0x10b0148, 0x4)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:285 +0x205
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processPodEvent(0xc0002fc140, 0xc0004eb350, 0x0, 0x0)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:275 +0x137
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents.func1(0xc000366120, 0xc0002fc140, 0xc00009ad20, 0xc000366000, 0xc000312190)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:109 +0x1ed
created by github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:105 +0x78
commented

Experiencing same issues mentioned above when i try to install es-operator @ AKS cluster v1.2.6.

Any suggestions?

commented

Same issue on a self hosted cluster.

time="2019-06-24T14:53:56Z" level=info msg="elasticsearch operator starting up!"
time="2019-06-24T14:53:56Z" level=info msg="Using Variables:"
time="2019-06-24T14:53:56Z" level=info msg="   enableInitDaemonset: true"
time="2019-06-24T14:53:56Z" level=info msg="   baseImage: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0"
time="2019-06-24T14:53:56Z" level=info msg="Using InCluster k8s config"
time="2019-06-24T14:53:56Z" level=info msg="SKIPPING: already exists "elasticsearchclusters.enterprises.upmc.com""
time="2019-06-24T14:53:58Z" level=info msg="Daemonset logging/elasticsearch-operator-sysctl already exist, skipping creation ..."
time="2019-06-24T14:53:58Z" level=info msg="Found 0 existing clusters "
time="2019-06-24T14:53:58Z" level=info msg="Watching for elasticsearch events..."
panic: runtime error: slice bounds out of range

goroutine 68 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processDataPodEvent(0xc000386480, 0xc000826dd0, 0x10b0148, 0x4)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:285 +0x205
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processPodEvent(0xc000386480, 0xc000826dd0, 0x0, 0x0)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:275 +0x137
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents.func1(0xc0001e6060, 0xc000386480, 0xc000367140, 0xc0001e67e0, 0xc0003ba0e0)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:109 +0x1ed
created by github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:105 +0x78
commented

Probably fixed by #303 in 0.4.0