Does Kogito Operator requires OpenShift?
vladimirfx opened this issue · comments
In other words - can it be used in 'plain' Kubernetes and if yes - with which functionality restrictions?
Hi @vladimirfx, Yes our kogito operator can run on plain Kubernetes. As of now in k8s you will not get functionality like Source-to-image (KogitoBuild) or automatic routes creation/exposed services other than that almost all things are compatible to run on kubernetes, You can find more information in our docs k8s section
Thank you!
I’ve reads docs already and can not make clear distinction of functionality available in openshift deployment vs plain k8s. On other side I saw many imports/references to openshift packages in sources.
Moreover docs contain deployment section only for OpenShift (section 3).
I think it will be useful to know (for all users) what functionality is not available for plain k8s:
- KogitoBuild
- Routes. What kind of routing is not work without OpenShift?
- Something else? (automatic infra deployment, incomplete cli etc)
This information is essential to make proper architectural decisions.
Hi @vladimirfx!
Many thanks for raising this question. I agree with you that the docs could simplify and clarify these questions. I'll bring this to the team responsible and try to work this out.
On other side I saw many imports/references to openshift packages in sources.
The import from Openshift packages is to support functionality from that platform. It does not interfere with the functionality of Kubernetes whatsoever. The code doesn't execute if it's not on the target platform.
Moreover docs contain deployment section only for OpenShift (section 3).
There is room for improvements, but we do have a section for Kubernetes https://docs.jboss.org/kogito/release/latest/html_single/#proc-kogito-deploying-on-kubernetes_kogito-deploying-on-openshift.
I think it will be useful to know (for all users) what functionality is not available for plain k8s:
I agree. To somewhat answer what you've raised:
KogitoBuild
We are abstracting the build feature in the target platform. For now, we only support s2i, but we have plans to leverage Tekton as the target build infrastructure. This way, the same build process will work on both.
Routes. What kind of routing is not work without OpenShift?
On Kubernetes, users have to expose the services themselves. We suggest using Ingress: https://github.com/kiegroup/kogito-cloud-operator/blob/master/examples/process-quarkus-example-ingress.yaml.
Something else?
We could clarify what is supported from the CLI perspective in the docs. For example, uploading artifacts to the cluster is not possible on Kubernetes since we do not support KogitoBuild.
Thanks again. I'll take this to the responsible for the documentation and work with them to improve based on your suggestions.
@vladimirfx, just for you to know, we put a plan together, and we will work on the documentation to clarify k8s vs. OCP usage with the Operator.
@vladimirfx, just for you to know, we put a plan together, and we will work on the documentation to clarify k8s vs. OCP usage with the Operator.
Thank you!
We are abstracting the build feature in the target platform. For now, we only support s2i, but we have plans to leverage Tekton as the target build infrastructure. This way, the same build process will work on both.
Tekton is out in cluster builder choice! We can be early adopters. Actually now we make Tekton pipelines to build Kogito projects in cluster by hand.
@vladimirfx, do you mind sharing your pipelines? I'm also working on an initiative offline to have some CI/CD recipes for Kogito in this repo: https://github.com/m88i/kogito-cicd-recipes
Also, are you using the Operator to deploy the images after building them with Tekton?
cc @msmagnanijr
@vladimirfx, do you mind sharing your pipelines?
I will try to extract some common part to share. Actually our build is not only Kogito project but some multistage composition of sources with process validation and status reporting. So probably our current project build is not implementable with KogitoBuild. But if POC will be successful Kogito will be used in other projects where KogitoBuild resource is sufficient. Other usage may be generate KogitoBuild resource as output from our complex pipeline and give Kogito Operator do the rest.
@ricardozanini, thank for idea of some thin local maven repo for speedup.
Also, are you using the Operator to deploy the images after building them with Tekton?
Yes, we are planned to use it. Full pipeline is not tested yet.
I will try to extract some common parts to share.
@vladimirfx, that would be amazing!
@ricardozanini, thank for idea of some thin local maven repo for speedup.
What do you mean? 🤔
Let me know if you need any help in your use case. The project is super new, and we are looking for use cases to improve it. Any help would be very welcome.
Here's the JIRA to track the changes in the documentation: https://issues.redhat.com/browse/KOGITO-4312
I should start working on it next week. Thanks for the heads up!
What do you mean? 🤔
Internal Nexus. Currently we do not use one.
Use case:
- [System analyst] develops some standardized BPMN process (with customized editor). Such standard process design has some predefined inputs and outputs.
- [System analyst] pushes process design to Git.
- [System analyst] marks some Git commit with tag (or something else named Git ref).
- [Tekton] detects relevant commit and start pipeline.
- [Tekton Pipeline] detects which of standard process design type is committed and pulls relevant "wrapper" Quarkus Kogito project.
- [Tekton Pipeline] combines BPMN design with wrapper project
- [Tekton Pipeline] build resulting project (including project/type specific validation)
- [Tekton Pipeline] on success publish resulting image in internal reg and necessary Kogito CRDs to current k8s cluster.
- [Tekton Pipeline] on success publish some business specific events about success build.
- [Tekton Pipeline] on fail publish some business specific events about failed build.
Actual pipeline a bit complicated than described but it is not relevant. I've interested to replace step 7 with KogitoBuild resource. Ideally I don't want to push resulting src from step 6 to Git but instead make it available as tgz archive accessible by HTTP url. Also I need detailed report on build fail because it later presented to [System analyst].
Does it make sense to cover such use case in KogitoBuild controller ?
Hey @vladimirfx! First of all, many thanks for this valuable information. Much appreciated!
I think you can use KogitoBuild in this scenario using something like:
apiVersion: app.kiegroup.org/v1beta1
kind: KogitoRuntime
metadata:
name: process-quarkus-example
spec:
replicas: 1
--
# here, we have an alternative way of building our fancy project, that's it from a local binary
# we can start a new build with "oc start-build process-quarkus-example --from-dir=target" from our project root
# both builds will share the same ImageStream, so every time we build a new version of the project, it will redeploy the KogitoRuntime counterpart
apiVersion: app.kiegroup.org/v1beta1
kind: KogitoBuild
metadata:
name: process-quarkus-example
spec:
runtime: quarkus
type: BinaryYour CI could then start the build with oc start-build <kogito-project-name> -f <archive.tgz>. The operator will configure the BuildConfigs for you and will deploy the resulting image as a KogitoRuntime object. You won't need to bother with image pushing.
I'll draw a CI based on your use case and add it to the recipes. I'll let you know once I have something to show. :)
EDIT: about the internal Nexus, I also maintain the Nexus Operator. It's a way to have and maintain Nexus servers within OpenShift/Kubernetes clusters. :)
Interesting...
Thoughts:
- No OpenShift, so no oc (this issue about that). EKS or custom based on Amazon EC2. This is not our choice.
- Image in in-cluster registry is good actually because requirement for auto scale. Default scale is 0. Activation with knative auto scale mechanics. This is because many prebuilded processes with uncertain activation scenarios - from 0 to 1000 per sec (does it scales with infinispan persistence?) and then to 0 again.
Thanks again for Nexus Operator!
No OpenShift, so no oc (this issue about that). EKS or custom based on Amazon EC2. This is not our choice.
To bring KogitoBuild support on k8s, we have two options: I can work on a Buildah implementation to build your images in-cluster or leverage Tekton (assuming it's installed in your k8s cluster). For now, we are considering the latter, as I said before.
Image in in-cluster registry is good actually because requirement for auto scale. Default scale is 0. Activation with knative auto scale mechanics. This is because many prebuilded processes with uncertain activation scenarios - from 0 to 1000 per sec (does it scales with infinispan persistence?) and then to 0 again.
Watch this JIRA. We will add support to deploy Knative services with KogitoRuntime soon. I believe we will have to test how the services will behave in such a scenario about persistence. But I'm optimistic. I don't think we will have too many problems.
@vladimirfx please follow up the JIRA as mentioned in the previous comment. If you have any issues, please open a JIRA: https://issues.redhat.com/projects/KOGITO/summary