Giters

testcontainers / testcontainers-scala

Docker containers for testing in scala

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Installing an elasticsearch plugin

LeonardMeyer opened this issue · comments

commented

Hi,

I'm trying to install this plugin but could not find a way to do that using ElasticsearchContainer unfortunately, so I tried this :

    GenericContainer.Def(
      "docker.elastic.co/elasticsearch/elasticsearch:7.6.2",
      exposedPorts = Seq(9200, 9300),
      env = Map("discovery.type" -> "single-node"),
      command = Seq("bin/elasticsearch-plugin install analysis-kuromoji")
    )

But this gives me :

12:14:25.566 [ScalaTest-run] INFO  🐳 [docker.elastic.co/elasticsearch/elasticsearch:7.6.2] - Container docker.elastic.co/elasticsearch/elasticsearch:7.6.2 is starting: 1b0636320e170c85549247f22bf937f5f003758be6231fea2b250e64ed978ae0
12:14:30.588 [ScalaTest-run] ERROR 🐳 [docker.elastic.co/elasticsearch/elasticsearch:7.6.2] - Could not start container
 org.testcontainers.shaded.org.awaitility.core.ConditionTimeoutException: Lambda expression in org.testcontainers.containers.GenericContainer: expected the predicate to return <true> but it returned <false> for input of <InspectContainerResponse(args=[bin/elasticsearch-plugin install analysis-kuromoji], config=ContainerConfig(attachStderr=false, attachStdin=false, attachStdout=false, cmd=[bin/elasticsearch-plugin install analysis-kuromoji], domainName=, entrypoint=[/usr/local/bin/docker-entrypoint.sh], env=[discovery.type=single-node, PATH=/usr/share/elasticsearch/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, ELASTIC_CONTAINER=true], exposedPorts=[9200/tcp, 9300/tcp], hostName=1b0636320e17, image=docker.elastic.co/elasticsearch/elasticsearch:7.6.2, labels={org.label-schema.build-date=2020-03-26T06:34:37.794943Z, org.label-schema.license=Elastic-License, org.label-schema.name=Elasticsearch, org.label-schema.schema-version=1.0, org.label-schema.url=https://www.elastic.co/products/elasticsearch, org.label-schema.usage=https://www.elastic.co/guide/en/elasticsearch/reference/index.html, org.label-schema.vcs-ref=ef48eb35cf30adf4db14086e8aabd07ef6fb113f, org.label-schema.vcs-url=https://github.com/elastic/elasticsearch, org.label-schema.vendor=Elastic, org.label-schema.version=7.6.2, org.opencontainers.image.created=2020-03-26T06:34:37.794943Z, org.opencontainers.image.documentation=https://www.elastic.co/guide/en/elasticsearch/reference/index.html, org.opencontainers.image.licenses=Elastic-License, org.opencontainers.image.revision=ef48eb35cf30adf4db14086e8aabd07ef6fb113f, org.opencontainers.image.source=https://github.com/elastic/elasticsearch, org.opencontainers.image.title=Elasticsearch, org.opencontainers.image.url=https://www.elastic.co/products/elasticsearch, org.opencontainers.image.vendor=Elastic, org.opencontainers.image.version=7.6.2, org.testcontainers=true, org.testcontainers.sessionId=cf4c96a0-7b54-4719-9ae3-3d752729bcd1}, macAddress=null, networkDisabled=null, onBuild=null, stdinOpen=false, portSpecs=null, stdInOnce=false, tty=false, user=, volumes=null, workingDir=/usr/share/elasticsearch, healthCheck=null), created=2022-02-14T11:14:25.219321575Z, driver=overlay2, execDriver=null, hostConfig=HostConfig(binds=[], blkioWeight=0, blkioWeightDevice=null, blkioDeviceReadBps=null, blkioDeviceWriteBps=null, blkioDeviceReadIOps=null, blkioDeviceWriteIOps=null, memorySwappiness=null, nanoCPUs=0, capAdd=null, capDrop=null, containerIDFile=, cpuPeriod=0, cpuRealtimePeriod=0, cpuRealtimeRuntime=0, cpuShares=0, cpuQuota=0, cpusetCpus=, cpusetMems=, devices=null, deviceCgroupRules=null, deviceRequests=null, diskQuota=null, dns=null, dnsOptions=null, dnsSearch=null, extraHosts=[], groupAdd=null, ipcMode=shareable, cgroup=, links=[], logConfig=LogConfig(type=json-file, config={}), lxcConf=null, memory=0, memorySwap=0, memoryReservation=0, kernelMemory=0, networkMode=default, oomKillDisable=null, init=null, autoRemove=false, oomScoreAdj=0, portBindings={9300/tcp=[Lcom.github.dockerjava.api.model.Ports$Binding;@4190bc8a, 9200/tcp=[Lcom.github.dockerjava.api.model.Ports$Binding;@47d023b7}, privileged=false, publishAllPorts=false, readonlyRootfs=false, restartPolicy=no, ulimits=null, cpuCount=0, cpuPercent=0, ioMaximumIOps=0, ioMaximumBandwidth=0, volumesFrom=[], mounts=null, pidMode=, isolation=null, securityOpts=null, storageOpt=null, cgroupParent=, volumeDriver=, shmSize=67108864, pidsLimit=null, runtime=runc, tmpFs=null, utSMode=, usernsMode=, sysctls=null, consoleSize=[0, 0]), hostnamePath=/var/lib/docker/containers/1b0636320e170c85549247f22bf937f5f003758be6231fea2b250e64ed978ae0/hostname, hostsPath=/var/lib/docker/containers/1b0636320e170c85549247f22bf937f5f003758be6231fea2b250e64ed978ae0/hosts, logPath=/var/lib/docker/containers/1b0636320e170c85549247f22bf937f5f003758be6231fea2b250e64ed978ae0/1b0636320e170c85549247f22bf937f5f003758be6231fea2b250e64ed978ae0-json.log, id=1b0636320e170c85549247f22bf937f5f003758be6231fea2b250e64ed978ae0, sizeRootFs=null, imageId=sha256:f29a1ee41030e3963026369105f3bee76d75fdecbeca07932ac054126be7bff9, mountLabel=, name=/interesting_joliot, restartCount=0, networkSettings=NetworkSettings(bridge=, sandboxId=d1d1031b42d493e31f978265b7dcf969ae1e61544ec1388e8c315cb28e23dc9c, hairpinMode=false, linkLocalIPv6Address=, linkLocalIPv6PrefixLen=0, ports={}, sandboxKey=/var/run/docker/netns/d1d1031b42d4, secondaryIPAddresses=null, secondaryIPv6Addresses=null, endpointID=, gateway=, portMapping=null, globalIPv6Address=, globalIPv6PrefixLen=0, ipAddress=, ipPrefixLen=0, ipV6Gateway=, macAddress=, networks={bridge=ContainerNetwork(ipamConfig=null, links=[], aliases=null, networkID=f889c45b0a9c5b006e34f562bb48215fddacddc98135a1b510ace0748bb75cd5, endpointId=, gateway=, ipAddress=, ipPrefixLen=0, ipV6Gateway=, globalIPv6Address=, globalIPv6PrefixLen=0, macAddress=)}), path=/usr/local/bin/docker-entrypoint.sh, processLabel=, resolvConfPath=/var/lib/docker/containers/1b0636320e170c85549247f22bf937f5f003758be6231fea2b250e64ed978ae0/resolv.conf, execIds=null, state=InspectContainerResponse.ContainerState(status=exited, running=false, paused=false, restarting=false, oomKilled=false, dead=false, pid=0, exitCode=1, error=, startedAt=2022-02-14T11:14:25.560553195Z, finishedAt=2022-02-14T11:14:25.564986783Z, health=null), volumes=null, volumesRW=null, node=null, mounts=[], graphDriver=GraphDriver(name=overlay2, data=GraphData(rootDir=null, deviceId=null, deviceName=null, deviceSize=null, dir=null)), platform=linux)> within 5 seconds.
	at org.testcontainers.shaded.org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:166) ~[testcontainers-1.16.2.jar:?]
	at org.testcontainers.shaded.org.awaitility.core.AbstractHamcrestCondition.await(AbstractHamcrestCondition.java:86) ~[testcontainers-1.16.2.jar:?]
	at org.testcontainers.shaded.org.awaitility.core.ConditionFactory.until(ConditionFactory.java:939) ~[testcontainers-1.16.2.jar:?]
        ...
12:14:30.620 [ScalaTest-run] ERROR 🐳 [docker.elastic.co/elasticsearch/elasticsearch:7.6.2] - Log output from the failed container:
/usr/local/bin/docker-entrypoint.sh: line 37: /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-kuromoji: No such file or directory

I'm not sure how this is all wired but /usr/share/elasticsearch/bin/elasticsearch-plugin is indeed at the root of my container and I managed to run it from the inside so I guess this is some kind of plumbing issue ?

I found this but this is the Java API. Is there anything similar maybe ?

Thanks

commented

Ugh, always when you post... Had to do it like this obviously :
command = Seq("bin/elasticsearch-plugin", "install", "analysis-kuromoji")

But now I get this error : /bin/bash: /dev/tcp/localhost/9200: Cannot assign requested address

commented

@dimafeng So I added a wait strategy like in the Java version and it seems fine :

  override val containerDef =
    GenericContainer.Def(
      "docker.elastic.co/elasticsearch/elasticsearch:7.6.2",
      exposedPorts = Seq(containerPort, 9300),
      env = Map("discovery.type" -> "single-node"),
      command = Seq("bin/elasticsearch-plugin", "install", "analysis-kuromoji"),
      waitStrategy = new HttpWaitStrategy()
        .forPort(9200)
        .forStatusCodeMatching(response => response == 200 || response == 401)
        .withStartupTimeout(JDuraton.ofMinutes(2))
    )

But now I get :

15:32:20.512 [ScalaTest-run] ERROR 🐳 [docker.elastic.co/elasticsearch/elasticsearch:7.6.2] - Log output from the failed container:
-> Installing analysis-kuromoji
-> Downloading analysis-kuromoji from elastic
[=================================================] 100%?? 
-> Installed analysis-kuromoji



Exception encountered when invoking run on a nested suite - Container startup failed
org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:336)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)
	at com.dimafeng.testcontainers.SingleContainer.start(Container.scala:47)
	at com.dimafeng.testcontainers.ContainerDef.start(ContainerDef.scala:14)
	at com.dimafeng.testcontainers.ContainerDef.start$(ContainerDef.scala:12)
	at com.dimafeng.testcontainers.GenericContainer$Def.start(GenericContainer.scala:105)
	at com.dimafeng.testcontainers.scalatest.TestContainerForAll.startContainers(TestContainerForAll.scala:31)
	at com.dimafeng.testcontainers.scalatest.TestContainerForAll.startContainers$(TestContainerForAll.scala:30)
	at com.dimafeng.testcontainers.scalatest.TestContainersForAll.run(TestContainersForAll.scala:41)
	at com.dimafeng.testcontainers.scalatest.TestContainersForAll.run$(TestContainersForAll.scala:37)
	at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
	at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1320)
	at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1314)
	at scala.collection.immutable.List.foreach(List.scala:431)
	at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1314)
	at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:993)
	at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:971)
	at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1480)
	at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:971)
	at org.scalatest.tools.Runner$.run(Runner.scala:798)
	at org.scalatest.tools.Runner.run(Runner.scala)
	at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:38)
	at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:25)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:329)
	... 25 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:525)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:331)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 26 more
Caused by: java.lang.IllegalStateException: Container exited with code 0
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:497)
	... 28 more

Seems like the install went fine but the container could not start and I don't have the error...

EDIT : Okay I think this is because of this. Overriding the commands overwrites the Dockerfile's command apparently. I ended up doing this even though I wanted to execute it only once for all my tests (container is reused)

withContainers { c =>
      c.execInContainer("bin/elasticsearch-plugin", "install", "analysis-kuromoji")
     ...
}