Plugin "pfsense-dns-haproxy-ingress-proxy" sets last host alias only
aded opened this issue · comments
Hi Travis,
when I add more than one ingress on my cluster (created via different Helm charts), I only find an host alias in unbound, the one created through the last ingress. I guess the plugin sets (not adds) the host aliases...?
My ingress controller is nginx-ingress.
TIA!
Can you share your config along with the sample ingresses?
Sure.
My kpc config:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: kube-system
name: kubernetes-pfsense-controller-config
data:
config: |
controller-id: "mycluster"
enabled: true
plugins:
metallb:
enabled: false
haproxy-declarative:
enabled: true
haproxy-ingress-proxy:
enabled: true
ingressLabelSelector:
ingressFieldSelector:
defaultFrontend: private-http
defaultBackend: private-http
# by default anything is allowed
#allowedHostRegex: "/.*/"
pfsense-dns-services:
enabled: false
pfsense-dns-ingresses:
enabled: false
ingressLabelSelector:
ingressFieldSelector:
#allowedHostRegex: "/.*/"
dnsBackends:
dnsmasq:
enabled: true
unbound:
enabled: true
pfsense-dns-haproxy-ingress-proxy:
enabled: true
#allowedHostRegex: "/.*/"
dnsBackends:
dnsmasq:
enabled: false
unbound:
enabled: true
frontends:
private-http:
hostname: private-http.cluster
public-https:
hostname: public-https.cluster
Both frontends have the right IP, I'm using only "private-http" anyway.
I deploy 2 nginx ingress controllers via-Helm, these are the values files:
controller:
ingressClass: "nginx-private"
kind: DaemonSet
service:
enableHttp: True
enableHttps: False
externalTrafficPolicy: Local
type: NodePort
and
controller:
ingressClass: "nginx-public"
kind: DaemonSet
service:
enableHttp: False
enableHttps: True
externalTrafficPolicy: Local
type: NodePort
Let's say I first deploy graylog (via-Helm again): the host I specify in Helm values is correctly added as host alias. Then I deploy another chart (prometheus-operator): I end up with the alertmanager host alias only, the last one specified in prometheus-operator's values.
Anyway if you can't reproduce the issue with a "standard" configuration, I'm definitely doing something wrong... :-)
OK, can you try out the same setup with dnsmasq by chance? This very well could be a bug..I'll see if I can get a sample setup going..
Any luck trying this with dnsmasq? I'll probably have a chance to dig into this a bit more in the next couple days..
Hi Travis, I will be able to try within tomorrow, I'll update you asap!
I've found the bug(s) and will have a new release shortly as an FYI.
This should be fixed in v0.2.0
. Thanks for the report!
Hi Travis,
thanks for your time: I apologize for the delay in responding, I was able to set up a test env right now.
It seems the issue is still there, both in DNS-Resolver and DNS-Forwarder.
Relevant, maybe useful, logs from controller:
2020-04-17T09:59:40+00:00 store successfully initialized
2020-04-17T09:59:40+00:00 waiting for ConfigMap kube-system/kubernetes-pfsense-controller-config to be present and valid
2020-04-17T09:59:46+00:00 controller config loaded/updated
2020-04-17T09:59:46+00:00 loading plugin haproxy-declarative
2020-04-17T09:59:46+00:00 loading plugin haproxy-ingress-proxy
2020-04-17T09:59:46+00:00 loading plugin pfsense-dns-haproxy-ingress-proxy
2020-04-17T09:59:47+00:00 plugin (haproxy-declarative): successfully reloaded HAProxy service
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
2020-04-17T09:59:54+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): successfully reloaded dnsmasq service
2020-04-17T10:00:01+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): successfully reloaded unbound service
2020-04-17T10:00:01+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): successfully reloaded DHCP service
2020-04-17T10:00:02+00:00 plugin (haproxy-ingress-proxy): successfully reloaded HAProxy service
2020-04-17T10:00:05+00:00 plugin (haproxy-declarative): successfully reloaded HAProxy service
2020-04-17T10:00:08+00:00 plugin (haproxy-declarative): successfully reloaded HAProxy service
2020-04-17T10:02:07+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): /apis/networking.k8s.io/v1beta1/namespaces/pdragon/ingresses/prometheus-operator-prometheus ADDED - 33143
2020-04-17T10:02:07+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): /apis/networking.k8s.io/v1beta1/namespaces/pdragon/ingresses/prometheus-operator-alertmanager ADDED - 33144
2020-04-17T10:02:07+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): /apis/networking.k8s.io/v1beta1/namespaces/pdragon/ingresses/prometheus-operator-grafana ADDED - 33145
2020-04-17T10:02:17+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): /apis/networking.k8s.io/v1beta1/namespaces/pdragon/ingresses/prometheus-operator-prometheus MODIFIED - 33332
2020-04-17T10:02:17+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): /apis/networking.k8s.io/v1beta1/namespaces/pdragon/ingresses/prometheus-operator-grafana MODIFIED - 33333
2020-04-17T10:02:17+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): /apis/networking.k8s.io/v1beta1/namespaces/pdragon/ingresses/prometheus-operator-alertmanager MODIFIED - 33334
2020-04-17T10:02:18+00:00 plugin (haproxy-ingress-proxy): creating frontend: pdragon-pdragon-prometheus-operator-prometheus
2020-04-17T10:02:18+00:00 plugin (haproxy-ingress-proxy): creating frontend: pdragon-pdragon-prometheus-operator-alertmanager
2020-04-17T10:02:18+00:00 plugin (haproxy-ingress-proxy): creating frontend: pdragon-pdragon-prometheus-operator-grafana
2020-04-17T10:02:19+00:00 plugin (haproxy-ingress-proxy): successfully reloaded HAProxy service
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
PHP Warning: Illegal string offset 'host' in phar:///usr/local/bin/kubernetes-pfsense-controller/src/KubernetesPfSenseController/Plugin/Utils.php on line 42
2020-04-17T10:02:25+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): successfully reloaded dnsmasq service
2020-04-17T10:02:32+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): successfully reloaded unbound service
2020-04-17T10:02:33+00:00 plugin (pfsense-dns-haproxy-ingress-proxy): successfully reloaded DHCP service
I'm using v0.2.0 (don't forget to change version in deployment.yml... :-D):
Normal Pulling 11m kubelet, kworker-3 Pulling image "docker.io/travisghansen/kubernetes-pfsense-controller:v0.2.0"
Normal Pulled 11m kubelet, kworker-3 Successfully pulled image "docker.io/travisghansen/kubernetes-pfsense-controller:v0.2.0"
When I deploy prometheus-operator (I deploy it only, as it enables 3 different hosts - alertmanager, grafana and prometheus itself), only one host is present as host alias, both in unbound and dnsmasq.
My pfSense version is 2.4.4-RELEASE-p3.
Anyway I will try a completely fresh install asap (I hope in the next few hours!) and I'll write you updates if any.
Bye!
BTW I noticed that the latest version is 0.2.1: tried it, same result and same logs.
Do you 1 ingress with multiple hosts or just multiple ingresses?
I have 2 ingresses as described in previous comments, but only one is used. These are my values for prometheus-operator Helm chart:
---
prometheusOperator:
createCustomResource: false
prometheus:
ingress:
annotations:
kubernetes.io/ingress.class: "nginx-private"
enabled: true
hosts: [ "prom.mon.test-current" ]
alertmanager:
ingress:
annotations:
kubernetes.io/ingress.class: "nginx-private"
enabled: true
hosts: [ "alerts.mon.test-current" ]
grafana:
ingress:
annotations:
kubernetes.io/ingress.class: "nginx-private"
enabled: true
hosts: [ "mon.test-current" ]
KPC config for haproxy-ingress-proxy:
haproxy-ingress-proxy:
enabled: true
ingressLabelSelector:
ingressFieldSelector:
defaultFrontend: private-http
defaultBackend: private-http
# by default anything is allowed
#allowedHostRegex: "/.*/"
KPC config for pfsense-dns-haproxy-ingress-proxy:
pfsense-dns-haproxy-ingress-proxy:
enabled: true
#allowedHostRegex: "/.*/"
dnsBackends:
dnsmasq:
enabled: true
unbound:
enabled: true
frontends:
private-http:
hostname: private-http.pdragon
public-https:
hostname: public-https.pdragon
private-5141:
hostname: private-5141.pdragon
And I end up with one alias (prom.mon.test-current) for hostname "private-http.pdragon".
Let me build a cluster from scratch and I'll update you if good news...
ok, I'm retesting as I likely still have something off with this. The correct rules are created in haproxy for all the ingresses yeah?
Yes, the pfSense's HAProxy is ok (frontends/backends), and if I manually add host aliases it works.
OK, I'm pretty sure I fixed your issue. Can you give v0.2.2
or latest
a try?
Hi Travis, it works like a charm! :-)
Going to close the issue...
Nice! Thanks for the report and patience!
Thanks to you for the great job and the controller! I might have more time in the next weeks to take a look at the code, so I might send a PR directly next time... Bye!