Kubernetes?
spacepluk opened this issue · comments
Did anybody manage to get this running on Kubernetes?
Thanks
This setup is working for me for a very simple deployment:
kind: Service
apiVersion: v1
metadata:
name: invoiceninja
labels:
app: invoiceninja
annotations:
external-dns.alpha.kubernetes.io/hostname: yourdomain
spec:
selector:
app: invoiceninja
ports:
- name: http
port: 80
targetPort: http
---
kind: ConfigMap
apiVersion: v1
metadata:
name: invoiceninja
labels:
app: invoiceninja
data:
nginx.conf: |
user www-data;
events {
worker_connections 768;
}
http {
upstream backend {
server 127.0.0.1:9000;
}
include /etc/nginx/mime.types;
default_type application/octet-stream;
gzip on;
gzip_disable "msie6";
server {
listen 80 default;
server_name _;
root /var/www/app/public;
index index.php;
charset utf-8;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location = /favicon.ico { access_log off; log_not_found off; }
location = /robots.txt { access_log off; log_not_found off; }
sendfile off;
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass backend;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
fastcgi_buffer_size 16k;
fastcgi_buffers 4 16k;
}
location ~ /\.ht {
deny all;
}
}
}
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: invoiceninja
labels:
app: invoiceninja
spec:
strategy:
type: Recreate
template:
metadata:
labels:
app: invoiceninja
spec:
containers:
- name: invoiceninja
image: invoiceninja/invoiceninja
ports:
- name: backend
containerPort: 9000
env:
- name: APP_ENV
value: 'production'
- name: APP_DEBUG
value: '0'
- name: APP_URL
value: 'yourdomain'
- name: APP_KEY
value: 'S0meRandomStringSomeRandomString'
- name: APP_CIPHER
value: 'AES-256-CBC'
- name: TRUSTED_PROXIES
value: '10.0.0.0/8'
- name: DB_TYPE
value: 'mysql'
- name: DB_USERNAME
value: 'root'
- name: DB_PASSWORD
value: 'ninja'
- name: DB_HOST
value: '127.0.0.1'
- name: DB_DATABASE
value: 'ninja'
- name: MAIL_HOST
value: mailserver
- name: MAIL_DRIVER
value: smtp
- name: MAIL_FROM_NAME
value: 'InvoiceNinja'
- name: MAIL_FROM_ADDRESS
value: 'finance@yourdomain'
volumeMounts:
- name: invoiceninja-shared
mountPath: /var/www/app/public
- name: invoiceninja-storage
mountPath: /var/www/app/storage
subPath: storage
- name: invoiceninja-storage
mountPath: /var/www/app/public/logo
subPath: logo
- image: mysql:5
name: mysql
args: ['--default-authentication-plugin=mysql_native_password']
env:
- name: MYSQL_DATABASE
value: 'ninja'
- name: MYSQL_ROOT_PASSWORD
value: 'ninja'
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: invoiceninja-storage
mountPath: /var/lib/mysql
subPath: mysql
- image: nginx:1
name: nginx
ports:
- name: http
containerPort: 80
volumeMounts:
- name: invoiceninja-shared
mountPath: /var/www/app/public
- name: invoiceninja-storage
mountPath: /var/www/app/storage
subPath: storage
- name: invoiceninja-storage
mountPath: /var/www/app/public/logo
subPath: logo
- name: invoiceninja
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
volumes:
- name: invoiceninja-shared
emptyDir: {}
- name: invoiceninja
emptyDir: {}
- name: invoiceninja-storage
emptyDir: {}
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: invoiceninja
annotations:
kubernetes.io/tls-acme: "true"
spec:
tls:
- secretName: invoiceninja-tls
hosts:
- yourdomain
rules:
- host: yourdomain
http:
paths:
- path: /
backend:
serviceName: invoiceninja
servicePort: 80
hello @spacepluk
thank you for sharing this.
Do you know how other project manage to add this in their repository ?
Hi. Thx @spacepluk for sharing this.
How about creating a helm chart with this? It will be super helpful and easy to install and test.
Thx
@lalop sorry I didn't see your message. Most projects I've seen just drop the yaml files in a deploy folder.
@botzill that's probably a good idea but I've never used helm myself :)
@spacepluk I'm doing some tests now. Will see how this is going and if all is OK, we can deploy one here: https://github.com/helm/charts. Will be good to have a review from you.
Thx.
@botzill What about helm chart for invoiceNinja, you planned to realise? I just research is any chart already available.
Hi,
I just create a helm chart to deploy invoice-ninja on kubernetes. You can find it here. I discovered invoice-ninja few days ago so probably some features are not working. I need feedbacks to improve the chart. So if someone is interested, please feel free to create issues on the repo.
Thx.
Hello, I can't manage the kubernetes part of invoice ninja, but if some of you are okay for maintaining it we may add an official chart to this repos
Since this is still the first hit when you google "invoiceninja kubernetes"... I figured I would share my updated config that works with v5 and the alpine images. I hope that helps somebody.
kind: Service
apiVersion: v1
metadata:
name: invoiceninja
labels:
app: invoiceninja
annotations:
external-dns.alpha.kubernetes.io/access: public
spec:
selector:
app: invoiceninja
ports:
- name: http
port: 80
targetPort: http
---
kind: ConfigMap
apiVersion: v1
metadata:
name: invoiceninja-env
labels:
app: invoiceninja
data:
APP_ENV: 'production'
APP_DEBUG: 'false'
APP_URL: 'https://invoice.yourdomain.com'
APP_KEY: 'S0meR4nd0mStr1ngS0meR4nd0mStr1ng'
APP_CIPHER: 'AES-256-CBC'
TRUSTED_PROXIES: '*'
DB_TYPE: 'mysql'
DB_STRICT: 'false'
DB_HOST1: '127.0.0.1'
DB_DATABASE1: 'ninja'
DB_USERNAME1: 'root'
DB_PASSWORD1: 'ninja'
MAIL_FROM_NAME: 'User Name'
MAIL_FROM_ADDRESS: 'finance@yourdomain.com'
MAIL_MAILER: 'smtp'
MAIL_PORT: '587'
MAIL_ENCRYPTION: 'TLS'
MAIL_HOST: 'smtp.mailgun.org'
MAIL_USERNAME: 'username'
MAIL_PASSWORD: 'password'
---
kind: ConfigMap
apiVersion: v1
metadata:
name: invoiceninja-cron-config
labels:
app: invoiceninja
data:
cron.sh: |
#!/bin/sh
echo "Early Entry"
cleanup ()
{
kill -s SIGTERM $!
exit 0
}
trap cleanup SIGINT SIGTERM
while :
do
sleep 60 ; cd /var/www/app/ && php artisan schedule:run;
done
---
kind: ConfigMap
apiVersion: v1
metadata:
name: invoiceninja-nginx-config
labels:
app: invoiceninja
data:
nginx.conf: |
user www-data;
events {
worker_connections 768;
}
http {
upstream backend {
server 127.0.0.1:9000;
}
include /etc/nginx/mime.types;
default_type application/octet-stream;
gzip on;
gzip_disable "msie6";
server {
listen 80 default;
server_name _;
root /var/www/app/public;
index index.php;
charset utf-8;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location = /favicon.ico { access_log off; log_not_found off; }
location = /robots.txt { access_log off; log_not_found off; }
sendfile off;
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass backend;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
fastcgi_param HTTPS 1;
fastcgi_buffer_size 16k;
fastcgi_buffers 4 16k;
}
location ~ /\.ht {
deny all;
}
}
}
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: invoiceninja
labels:
app: invoiceninja
spec:
strategy:
type: Recreate
selector:
matchLabels:
app: invoiceninja
template:
metadata:
labels:
app: invoiceninja
spec:
initContainers:
- name: volume-mount-hack
image: busybox
command: ["sh", "-c", "chown -R 1500:82 /var/www/app/public /var/www/app/storage"]
volumeMounts:
- name: invoiceninja-shared
mountPath: /var/www/app/public
- name: invoiceninja-storage
mountPath: /var/www/app/storage
subPath: storage
- name: invoiceninja-storage
mountPath: /var/www/app/public/storage
subPath: public-storage
containers:
- image: invoiceninja/invoiceninja:5.0.52
name: invoiceninja
ports:
- name: backend
containerPort: 9000
envFrom:
- configMapRef:
name: invoiceninja-env
volumeMounts:
- name: invoiceninja-shared
mountPath: /var/www/app/public
- name: invoiceninja-storage
mountPath: /var/www/app/storage
subPath: storage
- name: invoiceninja-storage
mountPath: /var/www/app/public/storage
subPath: public-storage
- image: invoiceninja/invoiceninja:5.0.52
name: cron
command: [ 'sh', '/usr/local/bin/cron.sh' ]
envFrom:
- configMapRef:
name: invoiceninja-env
volumeMounts:
- name: invoiceninja-cron-config
mountPath: /usr/local/bin/cron.sh
subPath: cron.sh
- name: invoiceninja-shared
mountPath: /var/www/app/public
- name: invoiceninja-storage
mountPath: /var/www/app/storage
subPath: storage
- name: invoiceninja-storage
mountPath: /var/www/app/public/storage
subPath: public-storage
- image: mysql:5
name: mysql
args: ['--default-authentication-plugin=mysql_native_password']
env:
- name: MYSQL_DATABASE
valueFrom:
configMapKeyRef:
name: invoiceninja-env
key: DB_DATABASE1
- name: MYSQL_ROOT_PASSWORD
valueFrom:
configMapKeyRef:
name: invoiceninja-env
key: DB_PASSWORD1
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: invoiceninja-storage
mountPath: /var/lib/mysql
subPath: mysql
- name: invoiceninja-storage
mountPath: /data
- image: nginx:1
name: nginx
ports:
- name: http
containerPort: 80
volumeMounts:
- name: invoiceninja-shared
mountPath: /var/www/app/public
- name: invoiceninja-storage
mountPath: /var/www/app/storage
subPath: storage
- name: invoiceninja-storage
mountPath: /var/www/app/public/storage
subPath: public-storage
- name: invoiceninja-nginx-config
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
volumes:
- name: invoiceninja-shared
emptyDir: {}
- name: invoiceninja-storage
persistentVolumeClaim:
claimName: invoiceninja-data-pvc
- name: invoiceninja-cron-config
configMap:
name: invoiceninja-cron-config
- name: invoiceninja-nginx-config
configMap:
name: invoiceninja-nginx-config
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: invoiceninja-data-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
#storageClassName: manual
#volumeName: invoiceninja-data
# ---
# apiVersion: v1
# kind: PersistentVolume
# metadata:
# name: invoiceninja-data
# labels:
# app: invoiceninja
# spec:
# capacity:
# storage: 1Gi
# volumeMode: Filesystem
# accessModes:
# - ReadWriteOnce
# persistentVolumeReclaimPolicy: Retain
# storageClassName: manual
# claimRef:
# name: invoiceninja-data-pvc
# namespace: invoiceninja # <---------- Make sure this matches your deployment
# awsElasticBlockStore:
# volumeID: vol-0123456789abcdef
# fsType: ext4
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: invoiceninja
annotations:
kubernetes.io/tls-acme: "true"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- invoice.yourdomain.com
secretName: invoiceninja-tls
rules:
- host: invoice.yourdomain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: invoiceninja
port:
number: 80
UPDATE: Added the missing cron container (fixes email sending). @turbo124 Thanks for helping!
UPDATE2: Added an example of persistent storage
@spacepluk, I have converted your sample to helm template
https://github.com/Saddamus/invoiceninja-helm
@spacepluk @Saddamus i've referenced this great work in the readme, thankyou!
Hi,
I'd be happy to see an up to date configuration example for Kubernetes 😄
It seems that this example and the referenced helm chart are both not updated anymore?
Is there an "official" page were the most recent version is tracked?
Regarding helm chart
I think that the only thing that may need to be updated is imagetag:
tag: "5.0.52", adjusting to latest one from dockerhub. The rest of env should work properly.
If there are some issues, im open to help and update the result on github.
Seems this is fixed now at https://github.com/invoiceninja/dockerfiles/tree/master/charts/invoiceninja ?
@Saddamus maybe you can link to that from yours as yours still comes up high in search.
Seems this is fixed now at https://github.com/invoiceninja/dockerfiles/tree/master/charts/invoiceninja ? @Saddamus maybe you can link to that from yours as yours still comes up high in search.
Done as requested.
I want to add that invoice ninja in docker has been painfully slow for me (v5) for any version higher than v5.2.19. This is with a blank install, as in blank container, blank database, etc. The initial page load for v5.2.19 is quick as one would expect.... v5.3-5.4 default pages are non-usable (http links in all-https page), and v5.5+ (including v5.7.x) take 2+ SECONDS to load the page initially and if anyone refreshes.
So for anyone reading this in the future, use v5.2.19 until the devs figure out the ridiculous performance issues...
Even adding things like "IS_DOCKER" defined as true (env variable) I see no difference... @spacepluk is the latest v5 slow for you too for initial page loads and page refreshes? (f5) and if not... what did you do to speed it up? Again nothing higher than v5.2.19 loads quickly for me no matter what I seem to throw at it so far... v5.2.19 loads very quickly.
btw thanks to @spacepluk and others in this thread, there's almost no kubernetes info for this docker image, the devs don't seem concerned about accurate documentation either, so this information has been utterly crucial to get operational, despite the load issues I still have (keeping me at v5.2.19).
So in the end I have to go with latest since my v4.5.50 will NOT migrate data to v5.2.19 >:| so I still have to deal with the obnoxious initial/refresh page speed... fortunately once it's loaded it's rather responsive.
I would like to add that these were some important details for success for my case:
Notice the change for the group id to 33:
- name: invoiceninja-hermes-volume-mount-hack
image: busybox
command: ["sh", "-c", "chown -R 1500:33 /var/www/app/public /var/www/app/storage"]
volumeMounts:
The underlying permanent storage is using smb.csi.k8s.io the SMB CSI from them, and these the relevant PV mount parameters I use, matching above too:
mountOptions:
- dir_mode=0770
- file_mode=0770
- uid=1500
- gid=33
- vers=3.0
Also note that TRUSTED_PROXIES="*" is a bad practice as it can lead to external websites impersonating to intercept credentials through embedding. Go look up Content Security Policy recommendations to see why this is an insecure configuration. I have it set to my LAN IP range very wide, since this runs in k8s and all IPs are dynamic in there (and I don't manage them at all, by choice).
I had the red triangle until I ran the update secret URL thing, and then refreshed the page
I'm also running mariadb in another deployment, instead of as a sidecar within the pod. That's more my choice really than a requirement for success.
The permission details above corrected the problem of not being able to see company logos. Now I can see them yay! And you will need to have that set correctly BEFORE you migrate any company data form v4! Otherewise you may have to purge company data and try migrating again (TAKE BACKUPS OF COURSE OF THE SOURCE SIDE!)
But yeah, initial page loads are like 4-7 seconds still...