use docker to run CI
sandys opened this issue · comments
Instead of running jobs directly, you should use docker.
It looks good, but it's not in the current scope of this plug-in because the current purpose of this plugin is just offering experience of CI on GitBucket without troublesome setup. We recommend to move to Jenkins if you like it and want to use it in your real projects.
I think the essence of using docker is to abstract the build from the server especially if a specific resource pool is required.
Though it will still come with its overhead which i think takezoe is trying to avoid.
My suggestion too,
what about sshing into a specified node. Here we will only deal with storing ssh key, node name and address.
I am not implying that docker is not okay. The overhead i mean is that, before you can use it , you must install docker in the same host as the gitbucket server which could still take up resources too. Then if you want to abstract out the job from the gitbucket server then you have to then create a swarm or kubernetes which will still come with their own learning curve.
But dont get me wrong , i think docker is awesome but i am just considering a light gitbucket
if you have to use ssh keys, you can manage as many as how many nodes only by saving the keys in the database then pulling it when you want to connect to any node and the nodes can still have docker installed.if required.
But the beautiful thing about gitbucket is its plugin system, we can use any of the two.or both.
Hey @sandys when you say docker i assume you mean swarm? Well pointing the jobs to a swarm endpoint would be fine too, in that case the gitbucket would only have to manage swarm endpoints according to users. I think that is the way gitlab does.
@sandys are you taking considerations too that many people too run gitbucket as a docker container. That means if you want to run jobs as a docker container WITHIN the gitbucket server container then you have to run it as a dind and the last i checked, dind has not had docker and git run together unless as a privileged user. Or is there a workaround ?
sorry for the delayed response. holiday travel!
@lavvy thanks for asking - this is pretty much a solved problem. Every CI runner actually runs as a docker container itself. Here's the official docs of Gitlab which goes into deep detail as well as multiple options to do this. This is actually a good thing and ensures safety all around.
https://docs.gitlab.com/ce/ci/docker/using_docker_build.html
just a quick update in case this was missed. gitlab-runner is run on every machine that needs to run the CI docker tasks - https://gitlab.com/gitlab-org/gitlab-runner and this spawns the actual docker images
@sandys Thanks for information. I will check it.
Anyway, docker is good solution for CI as you mentioned, but it's not in the scope of this plugin for now. One reason is that I don't have enough time to do it now. However if we want to make this plugin authentic CI tool in the future, we will need to make master-slave (or worker) model and consider docker support as well.
What i now do is
'''
scp - i id_rsa . root@anyserver.com:/tmp && ssh - i id_rsa root@anyserver.com && cd /tmp && make
'''
It can be as simple as this to abstract builds from the main server and scale out builds even without much overhead . It can be supported generically with just adding to settings where a user can add hosts and ssh key files (which i think has been supported for repositories).
Even if this is supported it will not change anything in still supporting docker.
I am coming from python and trying to learn scala but this scala is just unnecessarily kicking my butt lol, if not i would have sent a pull request long ago bcos i think it would be so easy to add this and with no overhead and can scale out(which is very important to us).
@lavvy Yes, that's an easiest, efficient and secure way to run build on a remote machine.
On the other hand, docker is an ideal solution to make independent environments for CI. Basically, this plugin doesn't depend any softwares other than JVM according to the policy of GitBucket. But I think it's nice if this plugin will support docker as an option.
Another way to get CI into gitbucket is to use an external CI runner. For instance, https://docs.gitlab.com/runner/
One advantage of this approach is it avoids issues with docker-in-docker and provisioning dependencies into the gitbucket instance. But I guess if gitbucket is already in a container then the security concerns are somewhat ameliorated.