Kubernetes/Cluster deployment

Question

Kubernetes/Cluster deployment

sk1p opened this issue 6 years ago · comments

Alexander Clausen commented 6 years ago

create a Docker image #484
create automation for installing libertem on a k8s cluster (helm?)
#106 Optimize handling of result tiles (added by @uellue)

See also #67 for auth related questions.

Edit: Removed HDFS aspects

Alexander Clausen · Answer 1 · Mon Nov 19 2018 17:14:11 GMT+0800 (China Standard Time)

For docker, see also: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/

Rahul Chandra · Answer 2 · Thu Feb 20 2020 17:48:14 GMT+0800 (China Standard Time)

https://stackoverflow.com/questions/26753030/how-to-build-docker-image-from-github-repository/39194765

https://docs.docker.com/docker-hub/builds/
Can @sk1p , we can think about creating the docker image directly through Github?

Alexander Clausen · Answer 3 · Thu Feb 20 2020 18:37:17 GMT+0800 (China Standard Time)

We would love to have a docker build and publish at the end of our CI pipeline. Currently, we are building (not publishing) a docker image as a step of the dist job, but in addition, we should publish it if all other jobs have succeeded, in the packaging stage (see .travis.yml).

I think using the automated building from docker hub means the docker image will be built independently from the CI fail/success status, right?

Rahul Chandra · Answer 4 · Mon Feb 24 2020 17:53:53 GMT+0800 (China Standard Time)

Yes, I also think so the docker hub build will be independent of CI fail/success status. Not quite sure about it but might we need to update the docker hub after every new build?.

@sk1p and @uellue I am moving forward with this https://docs.docker.com/docker-hub/builds/ and hope everything would be fine at the end.

Alexander Clausen · Answer 5 · Wed Feb 26 2020 05:36:40 GMT+0800 (China Standard Time)

Hmm, I'm not convinced that's the way to go. How do you prevent broken docker images from being pushed? If your answer is using autotest, how do you prevent having a completely separate test run from the rest of our pipeline? Feel free to play around with the build service, but you'll have to convince me to make it the official image 😁

On our road map, we have a migration to azure pipelines for CI, to consolidate our travis and appveyor builds into a single service. We already have the basics implemented in #565, it's mostly missing replacements for some travis specifics (basically travis-sphinx and updating the release script to migrate from the TRAVIS_* environment variables). Do you think that it may be possible to build the docker image from an azure pipeline job?