[BUG] Can't Get Real IP, getting Docker Bridge Gateway IP instead
xlionjuan opened this issue · comments
Describe the bug 🐛
Logs from Traefik:
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:27:06 New initialized mode:live
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:27:49 ServeHTTP ip:172.26.0.1 isTrusted:false
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:27:49 cache:GetDecision ip:172.26.0.1
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:27:49 ServeHTTP:getDecision ip:172.26.0.1 isBanned:false cache:miss
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:27:49 ServeHTTP:handleNoStreamCache ip:172.26.0.1 isBanned:true crowdsecQuery url:http://crowdsec:8080/v1/decisions?ip=172.26.0.1&banned=true, statusCode:403
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:28:14 ServeHTTP ip:172.26.0.1 isTrusted:false
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:28:14 cache:GetDecision ip:172.26.0.1
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:28:14 ServeHTTP:getDecision ip:172.26.0.1 isBanned:false cache:miss
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:28:14 ServeHTTP:handleNoStreamCache ip:172.26.0.1 isBanned:true crowdsecQuery url:http://crowdsec:8080/v1/decisions?ip=172.26.0.1&banned=true, statusCode:403
log from crowdsec:
time="16-10-2023 14:41:24" level=error msg="while fetching bouncer info: select bouncer: ent: bouncer not found: unable to query" ip=172.26.0.3
time="16-10-2023 14:41:24" level=info msg="172.26.0.3 - [Mon, 16 Oct 2023 14:41:24 UTC] \"GET /v1/decisions?ip=172.26.0.1&banned=true HTTP/1.1 403 630.628µs \"Go-http-client/1.1\" \""
time="16-10-2023 14:41:24" level=error msg="UnmarshalJSON : invalid character '.' after top-level value" line="172.26.0.1 - - [16/Oct/2023:14:41:24 +0000] \"GET / HTTP/2.0\" 403 742 \"-\" \"-\" 41 \"router-{deleted}@file\" \"http://error_pages_serv:80\" 6ms"
time="16-10-2023 14:41:24" level=warning msg="failed to run filter : invalid character '.' after top-level value (1:1)\n | UnmarshalJSON(evt.Parsed.message, evt.Unmarshaled, \"traefik\") in [\"\", nil]\n | ^" id=restless-sun name=child-crowdsecurity/traefik-logs stage=s01-parse
time="16-10-2023 14:41:25" level=error msg="while fetching bouncer info: select bouncer: ent: bouncer not found: unable to query" ip=172.26.0.3
time="16-10-2023 14:41:25" level=info msg="172.26.0.3 - [Mon, 16 Oct 2023 14:41:25 UTC] \"GET /v1/decisions?ip=172.26.0.1&banned=true HTTP/1.1 403 674.255µs \"Go-http-client/1.1\" \""
time="16-10-2023 14:41:25" level=error msg="UnmarshalJSON : invalid character '.' after top-level value" line="172.26.0.1 - - [16/Oct/2023:14:41:25 +0000] \"GET / HTTP/2.0\" 403 742 \"-\" \"-\" 42 \"router-{deleted}@file\" \"http://error_pages_serv:80\" 4ms"
time="16-10-2023 14:41:25" level=warning msg="failed to run filter : invalid character '.' after top-level value (1:1)\n | UnmarshalJSON(evt.Parsed.message, evt.Unmarshaled, \"traefik\") in [\"\", nil]\n | ^" id=restless-sun name=child-crowdsecurity/traefik-logs stage=s01-parse
time="16-10-2023 14:41:35" level=info msg="127.0.0.1 - [Mon, 16 Oct 2023 14:41:35 UTC] \"GET /v1/heartbeat HTTP/1.1 200 1.289397187s \"crowdsec/v1.5.4-e4dcdd25728b914823525f1efabf18d5c454902b\" \""
time="16-10-2023 14:41:42" level=error msg="while fetching bouncer info: select bouncer: ent: bouncer not found: unable to query" ip=172.26.0.3
time="16-10-2023 14:41:42" level=info msg="172.26.0.3 - [Mon, 16 Oct 2023 14:41:42 UTC] \"GET /v1/decisions?ip=172.26.0.1&banned=true HTTP/1.1 403 705.6µs \"Go-http-client/1.1\" \""
time="16-10-2023 14:41:42" level=error msg="UnmarshalJSON : invalid character '.' after top-level value" line="172.26.0.1 - - [16/Oct/2023:14:41:42 +0000] \"GET / HTTP/2.0\" 403 742 \"-\" \"-\" 44 \"router-{deleted}@file\" \"http://error_pages_serv:80\" 5ms"
time="16-10-2023 14:41:42" level=warning msg="failed to run filter : invalid character '.' after top-level value (1:1)\n | UnmarshalJSON(evt.Parsed.message, evt.Unmarshaled, \"traefik\") in [\"\", nil]\n | ^" id=restless-sun name=child-crowdsecurity/traefik-logs stage=s01-parse
time="16-10-2023 14:41:47" level=error msg="while fetching bouncer info: select bouncer: ent: bouncer not found: unable to query" ip=172.26.0.3
time="16-10-2023 14:41:47" level=info msg="172.26.0.3 - [Mon, 16 Oct 2023 14:41:47 UTC] \"GET /v1/decisions?ip=172.26.0.1&banned=true HTTP/1.1 403 637.982µs \"Go-http-client/1.1\" \""
��
-10-2023 14:41:47" level=error msg="UnmarshalJSON : invalid character '.' after top-level value" line="172.26.0.1 - - [16/Oct/2023:14:41:47 +0000] \"GET / HTTP/2.0\" 403 742 \"-\" \"-\" 45 \"router-{deleted}@file\" \"http://error_pages_serv:80\" 12ms"
time="16-10-2023 14:41:47" level=warning msg="failed to run filter : invalid character '.' after top-level value (1:1)\n | UnmarshalJSON(evt.Parsed.message, evt.Unmarshaled, \"traefik\") in [\"\", nil]\n | ^" id=restless-sun name=child-crowdsecurity/traefik-logs stage=s01-parse
Applications behind Traefik can get Real IP, whoani shows it can able get X-Forwarded-For and X-Real-Ip, accesslog can get real IP also.
cscli decisions add --ip 1.2.3.4 --duration 24h --reason "web bruteforce" this command from Crowdsec can also ban my IP
Expected behavior 👀
Get Real IP
Context 🔎
middlewares config:
http:
middlewares:
crowdsec:
plugin:
crowdsec-bouncer-traefik-plugin:
enabled: true
logLevel: DEBUG
updateIntervalSeconds: 60
defaultDecisionSeconds: 60
httpTimeoutSeconds: 10
crowdsecMode: live
crowdsecLapiKey: ######
#crowdsecLapiKeyFile: /etc/traefik/cs-privateKey-foo
crowdsecLapiHost: crowdsec:8080
crowdsecLapiScheme: http
crowdsecLapiTLSInsecureVerify: false
crowdsecCapiMachineId: login
crowdsecCapiPassword: password
crowdsecCapiScenarios:
- crowdsecurity/http-path-traversal-probing
- crowdsecurity/http-xss-probing
- crowdsecurity/http-generic-bf
forwardedHeadersTrustedIPs:
- 172.26.0.1 # <-- I've tried this or not tried this, but didn't solved.
clientTrustedIPs:
- 192.168.2.0/24
forwardedHeadersCustomName: X-Real-IP # <-- I've Tried ``X-Forwarded-For``, ``X-Real-IP`` or without this value.
redisCacheEnabled: false
redisCacheHost: "redis:6379"
redisCachePassword: password
redisCacheDatabase: "5"Version (please complete the following information):
- OS: Ubuntu Core 22
- Traefik version: 2.10.5
- Plugin version: 1.1.16
- Redis ? : N/A
To Reproduce
Steps to reproduce the behavior:
Hello @xlionjuan
I see that you are using live mode.
With this you don't have to set thoses variables (below) for CAPI (central API) because it means you are running a LAPI crowdsec container which does processing of your logs.
crowdsecCapiMachineId: login
crowdsecCapiPassword: password
crowdsecCapiScenarios:
- crowdsecurity/http-path-traversal-probing
- crowdsecurity/http-xss-probing
- crowdsecurity/http-generic-bfYou do have to configure crowdsec to enable the scenario you want for instance like this:
crowdsec:
image: crowdsecurity/crowdsec:v1.5.3
container_name: "crowdsec"
restart: unless-stopped
environment:
COLLECTIONS: crowdsecurity/traefik
CUSTOM_HOSTNAME: crowdsec
# We need to register one api key per service we will use
BOUNCER_KEY_TRAEFIK: FIXME-LAPI-KEY-1=
volumes:
- ./acquis.yaml:/etc/crowdsec/acquis.yaml:ro
- logs:/var/log/traefik:ro
- crowdsec-db:/var/lib/crowdsec/data/
- crowdsec-config:/etc/crowdsec/
labels:
- "traefik.enable=false"From this log and the last part statusCode:403, I think your connection to the crowdsec LAPI is not authenticated correctly.
DEBUG: CrowdsecBouncerTraefikPlugin: 2023/10/16 22:27:49 ServeHTTP:handleNoStreamCache ip:172.26.0.1 isBanned:true crowdsecQuery url:http://crowdsec:8080/v1/decisions?ip=172.26.0.1&banned=true, statusCode:403
Can you recrate a LAPI key and add it in the bouncer configuration ?
172.17.0.0/16 is usually the network range for docker.
if you see in the logs
ServeHTTP ip:172.26.0.1 isTrusted:false
It probably means the X-forwarded-for from the header is not picked up correctly
From Crowdsec logs:
time="16-10-2023 14:41:24" level=warning msg="failed to run filter : invalid character '.' after top-level value (1:1)\n | UnmarshalJSON(evt.Parsed.message, evt.Unmarshaled, \"traefik\") in [\"\", nil]\n | ^" id=restless-sun name=child-crowdsecurity/traefik-logs stage=s01-parse
It seams to receive some event but is not able to parse them.
Can you share your logging configuration of traefik ?
Did you add the parser for traefik ?
Installed with this env var for instance: COLLECTIONS: crowdsecurity/traefik
If you could provide your docker-compose (with the strict minimum) it could be helpful.
In the meantime you may start from this one which is supposed to work well and test it with something like Whoami.
Hello @xlionjuan
I see that you are using
livemode. With this you don't have to set thoses variables (below) for CAPI (central API) because it means you are running a LAPI crowdsec container which does processing of your logs.crowdsecCapiMachineId: login crowdsecCapiPassword: password crowdsecCapiScenarios: - crowdsecurity/http-path-traversal-probing - crowdsecurity/http-xss-probing - crowdsecurity/http-generic-bf
@mathieuHa Thank you, I only delete these lines and recreate LAPI key, and it can get the client's real IP again!
I didn't see "Mark as answer" button, maybe you can enable it?