Need documentation - cloudflare usage
GreenBlast opened this issue · comments
I had issue with setting cloudflare, managed to debug it by myself by looking on the code. Probably documentation needs to be added:
Cloudflare didn't work with the settings:
login=accountemail
password=api-token
But it did work with
login=token
password=api-token
Couldn't create a new DNS entry, just edit existing one, so I needed to create an entry manually, and then ddclient editing it.
I had several additional challenges on setting up with cloudflare we can add to the docs, or better, fix the underlying issues :)
- Default ddclient.conf commented out sections are incorrectly formatted for cloudflare and a few others. Variables appear newline delimited, not comma delimited with backslashes. Removing the
,and\characters fixes the error which shows up asWARNING: skipping host: subdomain.domain.tld: 'login=' is an invalid login. - Global API Keys are not supported properly, contrary to the default ddclient.conf comments. According to the cloudflare docs and some testing with curl, the proper authorization headers for global API keys are
X-Auth-Email: email@example.comandX-Auth-Key: globalAPIKeyHere, but ddclient instead sendsAuthorization: Bearer APIKeyHere, which only supports regular API keys. This has been noted in dependent projects here opnsense/plugins#2842. The workaround is to not use global API keys and just use a regular API key that has access to the zone you want DNS records updated in. This can be found in cloudflare's "My Profile" -> "API Tokens" sidebar -> "create API key" in the upper right.
I’ve been spending hours banging my head at trying to make ddclient work with CloudFlare. The support is just terrible, or the documentation.
I understand the commas and backslashes are useful in case you want to update multiple IPs, but that format just never worked for me. Every time, the script would try to look for an empty domain name. For example:
https://api.cloudflare.com/client/v4/zones/<redacted>/dns_records?type=AAAA&name=
(Note there is no value after name=.)
Turns out, if you use \ to escape a new line, it cannot be followed by comments.
Yet, the example config file has comments at the end of such lines.
I tried to remove the commas and backslashes like @BraxtonLowers suggested, which worked. I also tried to keep the commas and backslashes, but remove the comments following those backslashes, and that also worked.
Also, since I have to update both A and AAAA records with IPv4 & IPv6 addresses respectively, I had to change the following option:
use=web, web=<url-to-my-ip-page>
To the following:
usev4=webv4, web=<url-to-my-ip-page>
usev6=webv6, web=<url-to-my-ip-page>
Hope that helps.
Hi everyone,
A few updates:
- I haven't yet gotten to looking at the cloudflare implementation in-depth. I will be starting that now.
- The config now allows comments after a backslash (see 3c522a7).
Specifically for the config: The config format has quite a bit of accumulated cruft and the docs are currently lacking.
Until I can get around to updating those docs, here's a quick clarification of some details that came up in this issue:
- A backslash concatenates two lines together (and now supports trailing comments after the backslash character)
- Leading and trailing whitespace is stripped
- Almost all key/value pairs are separated via commas or whitespace. The only exception are keywords ending in
password, which as of now arepasswordandfw-password. - The special cased keywords (
password,fw-password):- Must have values that are either single-quoted or consist of a sequence of non-whitespace characters that does not start with a single quote
- Therefore a special-cased key/value pair can only be separated from the next key/value pair via comma if the value is quoted. Otherwise the comma gets interpreted as part of the value.
Just to clarify what @GreenBlast wrote above
When he says to not use this configuration syntax
login=accountemail
password=api-token
and instead use
login=token
password=api-token
- The string
accountemailis a standin for your actual email address likeuser@example.com - The string
api-tokenis a standin for the API token you provisioned in CloudFlare - The string
tokenis the literal wordtokenand is not a standin for anything
Was anyone able to get this to work? I'm running ddclient 3.8.3 with the following config:
# Configuration file for ddclient generated by debconf
#
# /etc/ddclient.conf
use=web
protocol=cloudflare
zone=<base-domain-here>
login=token
password=<cloudflare-api-token-not-key-single-quoted>
<fqdn-here>
Getting: file /var/cache/ddclient/ddclient.cache, line 3: Invalid Value for keyword 'ip' = ''
@jkapl I think that problem has more to do with the way that you're determining your IP address, not with your ability to set it on cloudflare.
But also, your protocol line should be one long line (or a line with \ at the end of each line).
Here's my full working config
usev4=webv4
webv4=googledomains
daemon=300
ssl=yes
protocol=cloudflare, \
zone=example.com, \
login=token, \
password=i-put-my-non-global-token-here \
foo.example.com
Thanks for the reply @gene1wood.
I think ultimately it may be related to my older version of ddclient - I'm running it on a Raspberry pi and stuck with 3.8.3 until the next stable Debian release. Looks like v3.10.0_1 had some major changes to Cloudflare DNS handling.
The formatting changes you suggested did help - but Cloudflare still rejects the update, I think perhaps because the older version of ddclient is calling an outdated version of the CloudflareAPI:
RECEIVE: <html><head><title>Current IP Check</title></head><body>Current IP Address: <ip-address-here></body></html>
INFO: setting IP address to <ip-address> for <correct-fqdn>
UPDATE: updating <fqdn>
CONNECT: www.cloudflare.com
CONNECTED: using SSL
SENDING: GET /api_json.html?a=rec_load_all&z=<zone-here>&email=token&tkn=<correct-api-token> HTTP/1.0
SENDING: Host: www.cloudflare.com
SENDING: User-Agent: ddclient/3.8.3
SENDING: Connection: close
SENDING:
RECEIVE: HTTP/1.1 403 Forbidden
RECEIVE: Date: Fri, 26 Jan 2024 19:08:15 GMT
RECEIVE: Content-Type: text/html; charset=UTF-8
older version of ddclient is calling an outdated version of the CloudflareAPI:
Yes, that's exactly what's happening, see #42 , which is available in 3.9.0
Worked! Manually upgraded to 3.10.0 and seeing the record updated in Cloudflare. Thanks @gene1wood !
Hi, I came across this issue building a docker image
# Use Alpine Linux as the base image
FROM alpine:latest
# Set noninteractive mode for apk
ENV APK_FLAGS="--no-cache --quiet --no-progress"
# Install necessary dependencies
RUN apk update && \
apk add $APK_FLAGS \
build-base \
perl \
perl-io-socket-ssl \
ca-certificates \
&& rm -rf /var/cache/apk/*
# Install ddclient
RUN apk update && \
apk add $APK_FLAGS \
ddclient \
&& rm -rf /var/cache/apk/*
# Command to run ddclient as a service
ENTRYPOINT ["ddclient", "-daemon=0", "-debug", "-verbose", "-noquiet", "-foreground"]
Before adding more complexity with a shell script to recognize patterns of multiple secrets (accounts and tokens/passwords) I try to test with a simple volume to ddclient.conf.
I try to update A records for 2 different dns services in the same conf: cloudflare and dynu.
Only cloudflare gives me issues and weirdly it seems the web page it finds to try to fetch the ip from is this:
https://github.com/jonsuh/hamburgers some burger menus. XD
although when I do it for dynu alone no problems, it updates the A record. Cloudflare alone or both together haves issues.
ddclient.conf
I tried to use the same conf listed by @gene1wood, but it does not work
#Configuration file for ddclient
#daemon=600
#Logging configuration
syslog=yes # Log messages to syslog
facility=daemon # Specify syslog facility
#Where Host Internet IP from ISP is checked
use=web, web=https://cloudflare.com/cdn-cgi/trace, web-skip='ip='
#Configuration for Cloudflare
protocol=cloudflare, \
zone=example.com, \
ssl=yes, \
server=www.cloudflare.com, \
login=token, \
password=Zone.Dns.Edit.Token? \
example.com
#Configuration for Dynu
protocol=dyndns2
ssl=yes
server=api.dynu.com
password=md5_passwd
login=email@example.com
example2.com
Is the Zone.Dns.Edit.Token the one I need?
@gene1wood
Sorry, I corrected it.
Is the Zone.Dns.Edit.Token the one I need?
The value in the password argument depends on how you've set it up in Cloudflare
Line 193 in f5a1a90
This is either your global API key, or an API token. If you are using an API token, it must have the permissions "Zone - DNS - Edit" and "Zone - Zone - Read". The Zone resources must be "Include - All zones".
So it's either the global API key or an API token, and which one it is affects the login argument
Only needed if you are using your global API key. If you are using an API token, set it to "token" (without double quotes).
Since you're using a login of token in your example, have you indeed provisioned an API token with those permissions (and aren't using the Global API key)?
I try
login=token
password=api-token
doing this to get the token
And if you add to the ddclient config
verbose=yes
What output do you get from ddclient when it attempts to update DNS in CloudFlare?
Many
ddclient | RECEIVE: ....*I won't copy paste, it is a lot*
Midway I can see this:
ddclient | RECEIVE: * Hamburgers
ddclient | RECEIVE: * @description Tasty CSS-animated hamburgers
ddclient | RECEIVE: * @author Jonathan Suh @jonsuh
ddclient | RECEIVE: * @site https://jonsuh.com/hamburgers
ddclient | RECEIVE: * @link https://github.com/jonsuh/hamburgers
then a lot more gibberish...
**
The error I get is: ddclient | FAILED: updating example.com: Could not connect to www.cloudflare.com.
I'd recommend, to simplify your situation and identify the problem(s)
- Change your config to use a known working simple method to obtain the IP and see if that solves your hamburgers problem. Currently you're using
use=web, web=https://cloudflare.com/cdn-cgi/trace, web-skip='ip='
Change it to something that will work for sure
usev4=webv4
webv4=googledomains
And see if that solves the hamburgers part of the problem.
If so
- Remove the
dyndns2protocol section so that you're only updating CloudFlare to simplify the case.
The sections you redacted as giberish from the output may contain the details of the cause of the problem.
@gene1wood
Thank you for your time and replies. Sorry if I haven't been clear. I isolated both dns services already. DYNU works with the address I fetch the ip from. Cloudflare doesn't. I shared an example with both together, but I commented the DYNU part out.
Now I changed ddclient.conf to that taking out the comments as well.
# Configuration file for ddclient
# daemon=600
# Logging configuration
syslog=yes # Log messages to syslog
facility=daemon # Specify syslog facility
# Where Host Internet IP from ISP is checked
usev4=webv4
webv4=googledomains
# Configuration for Cloudflare
protocol=cloudflare
zone=example.com
ssl=yes
server=www.cloudflare.com
login=token
password=the-token
example.com
And looking at the messages, it actually gets an ip either from my
use=web, web=https://cloudflare.com/cdn-cgi/trace, web-skip='ip='
The problem is mainly that it can't connect to cloudflare with the token I provide. :(
From the screenshot I provided, is it the right config for it?
So in this example you give, your cloudlfare lines are formatted incorrectly, have an incorrect argument, and an incorrect value.
Your example shows
protocol=cloudflare
zone=example.com
ssl=yes
server=www.cloudflare.com
login=token
password=the-token
example.com
but it should be
protocol=cloudflare, \
zone=example.com, \
login=token, \
password=the-token \
example.com
Here I've added the , \ at the end of the line because this is supposed to be a single line for the argument protocol
I've also removed ssl because that's not an argument for the protocol line, that's a ddclient argument and can't go here.
I've also removed the server line because www.cloudflare.com isn't the correct value. That was the server before the version change I mention above
Oh thank you for your patience! I kind of lost track of setting it as you showed through the discussion. I am a bit tired. I apologize!
It works with an isolated config for cloudflare and also + dynu! after testing :)
For anyone with cloudflare and another dns service on dyndns protocol to update:
# Configuration file for ddclient
daemon=600
# Logging configuration
syslog=yes # Log messages to syslog
facility=daemon # Specify syslog facility
# Where Host Internet IP from ISP is checked
usev4=webv4
webv4=googledomains
ssl=yes
protocol=cloudflare, \
zone=example.com, \
login=token, \ #(written as token)
password=the-token \ #(see screenshot to generate it)
example.com
# Configuration for Dynu
protocol=dyndns2
ssl=yes
server=api.dynu.com
password=md5_passwd
login=email@example.com
example2.com
Screenshot, token to generate for cloudflare:
Is this the official documentation? https://ddclient.net/ I did not see anything about usev4 or that I could use googledomains. Although I am pretty sure the whole issue was the syntax, not the website I was fetching my WAN Ip from. I still don't understand why I was getting some random link to github hamburger menus.
Thank you again!
Cool. It does look like your dyndns2 config may also be wrong.
You've got
# Configuration for Dynu
protocol=dyndns2
ssl=yes
server=api.dynu.com
password=md5_passwd
login=email@example.com
example2.com
and it should probably be
# Configuration for Dynu
protocol=dyndns2, \
password=md5_passwd, \
login=email@example.com \
example2.com
because server currently defaults to members.dyndns.org instead of api.dynu.com
and ssl shouldn't be in there either (it's separate)
for dynu, I refered to this doc:
https://www.dynu.com/DynamicDNS/IPUpdateClient/DDClient
because
servercurrently defaults tomembers.dyndns.orginstead ofapi.dynu.com
But my dns service is with dynu, my understanding is that the server is the dns service and the protocol they use is dyndns2, so I need to specify api.dynu.com as the server, no?
and
sslshouldn't be in there either (it's separate)
I was actually wondering if ssl=yes was a global setting and not specific to a dns service to update? SO I should add it once at the top of the ddclient.conf?
for dynu, I refered to this doc:
Interesting, I'm not sure which one is right, the dynu doc or the ddclient examples / code comments
Lines 72 to 79 in f5a1a90
Lines 3899 to 3903 in f5a1a90
But my dns service is with dynu, my understanding is that the server is the dns service and the protocol they use is dyndns2, so I need to specify api.dynu.com as the server, no?
Ah sorry I didn't understand, you're right.
I was actually wondering if ssl=yes was a global setting and not specific to a dns service to update? SO I should add it once at the top of the ddclient.conf?
My understanding is that ssl is global and not related to the protocol setting
When I look at the examples in this repo for using ssl, it's always it's own argument, distinct from protocol
Some notes on confusion around the ssl setting
I don't have my github account set on my phone yet. I just started being active here. I was busy with other matters outside the computing world. Thank you for the reply!
My understanding is that ssl is global and not related to the protocol setting
When I look at the examples in this repo for using ssl, it's always it's own argument, distinct from protocol
I will change my dynu conf settings to reflect the ddclient one and see how it behaves. The most up to date is probably here.
Some notes on confusion around the ssl setting
I was wondering why having it ssl=yes had anything to do with problems I had fetching my wan IP with http://checkip.dyndns.org/, hence why I switched with a https link I got from cloudflare: https://cloudflare.com/cdn-cgi/trace, it idi work, and would probably still work with my cloudflare now fixed, but I will stick with usev4=webv4 webv4=googledomains. It seems more reliable.
Now that I can load a conf successfully. I am working on 3 things to add to my docker image.
-A wrapper script to interpret environment variables declared ${env_var} or ${env_var_FILE} to values depending if they have to be loaded from files as secrets /run/secrets/file_name or directly substituted. ddclient.conf doesn't seem to be able to substitute env var by default.
-How to get logs out of ddclient to link it to a volume. So far I was using this doc https://ddclient.net/usage.html to help me figure out how to setup the conf file, but it does not seem super accurate, I will use the github repo. syslog=yes and facility=daemon doesn't to give me any kind of output in /var/log/syslog.
-I should also make sure of the release I use on the base image (Alpine Linux, right now I apk add whatever is available) or figure a way to fetch the last release from github directly. I am kind of new to imaging with docker and linux as a whole.
I know what I just mentioned is not part of the topic here. Just letting you know if you have infos/time, you could pm me. Thank you for all the help! Eventually I will link my project on github.
My understanding is that ssl is global and not related to the protocol setting
When I look at the examples in this repo for using ssl, it's always it's own argument, distinct from protocolI will change my dynu conf settings to reflect the ddclient one and see how it behaves. The most up to date is probably here.
I was wondering why having it ssl=yes had anything to do with problems I had fetching my wan IP with http://checkip.dyndns.org/, hence why I switched with a https link I got from cloudflare: https://cloudflare.com/cdn-cgi/trace, it idi work, and would probably still work with my cloudflare now fixed, but I will stick withusev4=webv4 webv4=googledomains. It seems more reliable.
Note: webv4=googledomains is being deprecated. See #622 for more info.
Now that I can load a conf successfully. I am working on 3 things to add to my docker image. -A wrapper script to interpret environment variables declared
${env_var}or${env_var_FILE}to values depending if they have to be loaded from files as secrets /run/secrets/file_name or directly substituted. ddclient.conf doesn't seem to be able to substitute env var by default.
There is a mechanism for this as of v3.11.0. See here
Note:
webv4=googledomainsis being deprecated. See #622 for more info.
I guess I will go back to this then...
use=web, web=https://cloudflare.com/cdn-cgi/trace, web-skip='ip='
There is a mechanism for this as of v3.11.0. See here
I already finished the wrapper script, I load it from the Entrypoint of the Dockerfile and it works for any kind of declarations ${} and dynamically knows to load either from a file or environment variable depending if it ends with _FILE in their names. I will check the link though. Thank you.
I am still working on getting logs from syslog though. I think the reason I don't get anythign when I try to set a volume is because by default the base Alpine Linux Image does not have syslog enabled and/or installed.
Don't have enough time to deep dive here but this config works for me to update both ipv6 and ipv4:
usev4=webv4, webv4=https://api.ipify.org
usev6=webv6, webv6=https://api6.ipify.org
protocol=cloudflare, \
zone=ZONE, \
login=token, \
password='API_TOKEN', \
DOMAIN_NAME
Replace the variables in CAPITALIZED_SNAKE_CASE with your own config.
Removing comments was the trick for me.
With comments in the config file, ddclient fails at building dns_records GET:
SENDING: request=GET
SENDING: url="https://api.cloudflare.com/client/v4/zones/xxxxxxxxxxxx/dns_records?type=A&name=\\"
SENDING: header="Content-Type: application/json"
And receives no records.
RECEIVE: {"result":[],"success":true,"errors":[],"messages":[],"result_info":{"page":1,"per_page":100,"count":0,"total_count":0,"total_pages":1}}
FAILED: updating \: Cannot set IPv4 to 186.28.xx.xxx No 'A' record at Cloudflare
Without comments ddclient builds correctly the dns_recods GET:
SENDING: request=GET
SENDING: url="https://api.cloudflare.com/client/v4/zones/xxxxxxxxxxxx/dns_records?type=A&name=ddns.xxx.xxx"
SENDING: header="Content-Type: application/json"
And receives:
RECEIVE: {"result":{"id":"xxxxxxxxxx","zone_id":"xxxxxxxxxx","zone_name":"xxx.xxx","name":"ddns.xxx.xxx","type":"A","content":"186.28.xxx.xxx","proxiable":true,"proxied":false,"ttl":60,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false},"comment":null,"tags":[],"created_on":"2024-03-26T21:44:33.527931Z","modified_on":"2024-03-26T22:14:23.007728Z"},"success":true,"errors":[],"messages":[]}
SUCCESS: updating ddns.xxx.xxx: IPv4 address set to 186.28.xxx.xxx
Config with comments:
protocol=cloudflare, \
zone=domain.tld, \
ttl=1, \
login=your-login-email, \ # Only needed if you are using your global API key. If you are using an API token, set it to "token" (wihtout double quotes).
password=APIKey \ # This is either your global API key, or an API token. If you are using an API token, it must have the permissions "Zone - DNS - Edit" and "Zone - Zone - Read". The Zone resources must be "Include - All zones".
domain.tld,my.domain.tld
Without:
protocol=cloudflare, \
zone=domain.tld, \
ttl=1, \
login=your-login-email, \
password=APIKey \
domain.tld,my.domain.tld
Removing comments was the trick for me.
Thank you for this! I edited the default /usr/local/etc/ddclient.conf config, left everything in, but the one thing tripping it up was the comments after the cloudflare login and password. It now updates successfully.
Also having trouble to getting my CloudFlare DNS to work
cloudflare - NAS DOMAIN] error receiving ZoneID [[{"code": 6003, "message": "Invalid request headers", "error_chain": [{"code": 6111, "message": "Invalid format for Authorization header"}]}]]
Just started with OpnSense (before PfSense)
So I have the plugin ddclient, and need to edit the conf file from ddclient ....to what exaclty