icholy/digest auth failed where curl --digest succeeds
elwhite321 opened this issue · comments
As the title states, the icholy/digest auth fails where a curl --digest command succeeds.
The curl command with login removed:
curl -v -X PUT -H 'X-CSRF: x' -H "Accept: application/json" --data 'value=true' --digest 'http://***:***@192.168.0.100/restapi/relay/outlets/=0,1,2/state/'
Here is the Authorization header from curl's second, successful request with username and response removed:
Authorization: Digest username="*****", realm="DLI LPC92601002528", nonce="G/QIGjhJANuvwqcO", uri="/restapi/relay/outlets/=0,1,2/state/", cnonce="NjUxNzA5ZDQ0OGI1NDRlNzFmYTRiYTllNzJlMDdlYjM=", nc=00000001, qop=auth, response="************", opaque="MyhQh+WfWI37Ou00", algorithm="MD5"
Now with icholy/digest, using the README example to set username and password:
client := &http.Client{
Transport: &digest.Transport{
Username: "*********",
Password: "**********",
},
}
req, err := http.NewRequest(http.MethodPut,
"http://192.168.0.100/restapi/relay/outlets/=0,1,2/state",
strings.NewReader("value=true"))
if err != nil { return }
req.Header.Add("Accept", "application/json")
req.Header["X-CSRF"] = []string{"x"}
res, _ := client.Do(req)
fmt.Println(res)
And here is the request from the digest.Transport's second request to the server with the digest username and response removed:
&{PUT http://192.168.0.100/restapi/relay/outlets/=0,1,2/state HTTP/1.1 1 1 map[Accept:[application/json] Authorization:[Digest username="**********", realm="DLI LPC92601002528", nonce="mn+ExiOvWOnZuwHN", uri="/restapi/relay/outlets/=0,1,2/state", response=***************", algorithm=MD5, cnonce="6d00de0791f29f8c509d218b83680979", opque="MOJIFcQn52wT0EYX", qop=auth, nc=00000001] X-CSRF:[x]] {0xc000282060} 0x123ef80 10 [] false 192.168.0.100 map[] map[] <nil> map[] <nil> <nil> <nil> 0xc0000ba008}}
This request gives a 401 response with a new Www-Authenticate header.
To rule out the request body being the issue, the same curl above with garbage --data still authenticates with a 207 response.
Please let me know if there is any more information I can provide. I will update this as I explore the issue.
Edit: here is an example 401 response from the server via curl:
< HTTP/1.1 401 Unauthorized
< X-Content-Type-Options: nosniff
< Connection: close
< Vary: origin, accept, prefer
< Cache-Control: max-age=0, private, must-revalidate
< Content-Security-Policy: frame-ancestors 'self';default-src 'self';media-src 'none';img-src 'self' data:;object-src 'none'
< X-Frame-Options: sameorigin
< Accept-Ranges: dli-depth
< WWW-Authenticate: Digest algorithm="MD5", opaque="mNLjyiEsUhWpGkNB", qop="auth,auth-int", realm="DLI LPC92601002528", nonce="3U07AmOG2+OW9XOq"
< Content-Type: application/json
and the same via golang:
&{401 Unauthorized 401 HTTP/1.1 1 1 map[Accept-Ranges:[dli-depth] Cache-Control:[max-age=0, private, must-revalidate] Content-Security-Policy:[frame-ancestors 'self';default-src 'self';media-src 'none';img-src 'self' data:;object-src 'none'] Content-Type:[application/json] Vary:[origin, accept, prefer] Www-Authenticate:[Digest algorithm="MD5", opaque="MOJIFcQn52wT0EYX", qop="auth,auth-int", realm="DLI LPC92601002528", nonce="mn+ExiOvWOnZuwHN"] X-Content-Type-Options:[nosniff] X-Frame-Options:[sameorigin]] 0xc000024080 -1 [] true false map[] 0xc0000f0100 <nil>}
I suspect this endpoint requires auth-int QOP (quality of protection) for PUT/POST requests which this package doesn't implement. The challenge you posted advertises that it supports both auth and auth-int (this parameter: qop="auth,auth-int"). So the library chooses to use auth.
WWW-Authenticate: Digest algorithm="MD5", opaque="mNLjyiEsUhWpGkNB", qop="auth,auth-int", realm="DLI LPC92601002528", nonce="3U07AmOG2+OW9XOq"
You can check which QOP curl is using by looking at the request headers in wireshark or a similar tool. I'm reading the docs and I don't see an explicit requirement for auth-int, but that doesn't really mean anything. They're technically not RFC compliant by not accepting auth qop if their challenge advertises support for it. Let me know what you find out.
Thanks for the quick reply! And you found the DLI docs!
I did notice curl has qop="auth" in the Authorization header. From rfc2617, this means curl is applying the "auth" level of quality of protection, so I assumed auth was acceptable.
I used tcpdump to get the HTTP headers. Here is the output:
Host: 192.168.0.100
Authorization: Digest username="**********", realm="DLI LPC92601002528", nonce="TWGyyYf2UHIYkXBN", uri="/restapi/relay/outlets/=0,1,2/state/", cnonce="YzExYzc1ZWFhYzJiNGYwNDVhODJiZGQzZDY1YWM3NTM=", nc=00000001, qop=auth, response="***************************", opaque="bSVu23yCdV7mrPqM", algorithm="MD5"
User-Agent: curl/7.54.0
X-CSRF: x
Accept: application/json
Content-Length: 11
Content-Type: application/x-www-form-urlencoded
value=false
I did notice the content is urlencoded. I'll be looking into this now
Edit: urlencoded does not fix the issue.
@elwhite321 in that case, this sounds like a bug on my end. Would you be able to give me an unredacted exchange (pcap or plaintext) of the request/response headers curl successfully authenticates with? I imagine you can temporarily switch the username/password for the endpoint.
Yes, no problem!
Here is the curl command:
curl -v -X PUT -H 'X-CSRF: x' -H "Accept: application/json" --data 'value=false' --digest 'http://icholy:digest@192.168.0.100/restapi/relay/outlets/=0,1,2/state/'
And the full output:
* Trying 192.168.0.100...
* TCP_NODELAY set
* Connected to 192.168.0.100 (192.168.0.100) port 80 (#0)
* Server auth using Digest with user 'icholy'
> PUT /restapi/relay/outlets/=0,1,2/state/ HTTP/1.1
> Host: 192.168.0.100
> User-Agent: curl/7.54.0
> X-CSRF: x
> Accept: application/json
> Content-Length: 0
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 401 Unauthorized
< X-Content-Type-Options: nosniff
< Connection: close
< Vary: origin, accept, prefer
< Cache-Control: max-age=0, private, must-revalidate
< Content-Security-Policy: frame-ancestors 'self';default-src 'self';media-src 'none';img-src 'self' data:;object-src 'none'
< X-Frame-Options: sameorigin
< Accept-Ranges: dli-depth
< WWW-Authenticate: Digest algorithm="MD5", opaque="wRtIEgb/X9z7XXAT", qop="auth,auth-int", realm="DLI LPC92601002528", nonce="NZAeQHhoCNifFjFa"
< Content-Type: application/json
<
* Closing connection 0
* Issue another request to this URL: 'http://icholy:digest@192.168.0.100/restapi/relay/outlets/=0,1,2/state/'
* Hostname 192.168.0.100 was found in DNS cache
* Trying 192.168.0.100...
* TCP_NODELAY set
* Connected to 192.168.0.100 (192.168.0.100) port 80 (#1)
* Server auth using Digest with user 'icholy'
> PUT /restapi/relay/outlets/=0,1,2/state/ HTTP/1.1
> Host: 192.168.0.100
> Authorization: Digest username="icholy", realm="DLI LPC92601002528", nonce="NZAeQHhoCNifFjFa", uri="/restapi/relay/outlets/=0,1,2/state/", cnonce="MzI1MWE0MDI1MzEyOWQ2M2U1YjM1OGZiNWMwZWNiYjA=", nc=00000001, qop=auth, response="9e0d2169b41cbb504a58995e08b10eb1", opaque="wRtIEgb/X9z7XXAT", algorithm="MD5"
> User-Agent: curl/7.54.0
> X-CSRF: x
> Accept: application/json
> Content-Length: 11
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 11 out of 11 bytes
< HTTP/1.1 207 Responses from multiple resources follow
< X-Content-Type-Options: nosniff
< Connection: close
< Vary: origin, accept, prefer
< Cache-Control: max-age=0, private, must-revalidate
< Link: </restapi/relay/outlets/0/state/>; rel="item"
< Link: </restapi/relay/outlets/1/state/>; rel="item"
< Link: </restapi/relay/outlets/2/state/>; rel="item"
< Content-Security-Policy: frame-ancestors 'self';default-src 'self';media-src 'none';img-src 'self' data:;object-src 'none'
< X-Frame-Options: sameorigin
< Accept-Ranges: dli-depth
<
* Closing connection 1
I updated my golang code to reflect as well:
client := &http.Client{
Transport: &digest.Transport{
Username: "icholy",
Password: "digest",
},
}
req, err := http.NewRequest(http.MethodPut,
"http://192.168.0.100/restapi/relay/outlets/=0,1,2/state?value=true", strings.NewReader("value=true"))
if err != nil { return }
req.Header.Add("Accept", "application/json")
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
req.Header["X-CSRF"] = []string{"x"}
res, _ := client.Do(req)
fmt.Println(res)
The first RoundTrip from the Transport, request and then response:
&{PUT http://192.168.0.100/restapi/relay/outlets/=0,1,2/state?value=true HTTP/1.1 1 1
map[Accept:[application/json] Content-Type:[application/x-www-form-urlencoded] X-CSRF:[x]]
{0xc00012a2e0} 0x123ef80 10 [] false 192.168.0.100 map[] map[] <nil> map[] <nil> <nil> <nil> 0xc000136008}
&{401 Unauthorized 401 HTTP/1.1 1 1 map[Accept-Ranges:[dli-depth] Cache-Control:[max-age=0, private, must-revalidate] Content-Security-Policy:[frame-ancestors 'self';default-src 'self';media-src 'none';img-src 'self' data:;object-src 'none'] Content-Type:[application/json] Vary:[origin, accept, prefer]
Www-Authenticate:[Digest algorithm="MD5", opaque="YVjDroJeBJTjdrh/", qop="auth,auth-int", realm="DLI LPC92601002528", nonce="jiviS2HZ+nV2E6yN"] X-Content-Type-Options:[nosniff] X-Frame-Options:[sameorigin]] 0xc00021c040 -1 [] true false map[] 0xc00016e100 <nil>}
And finally the second trip:
&{PUT http://192.168.0.100/restapi/relay/outlets/=0,1,2/state?value=true HTTP/1.1 1 1 map[Accept:[application/json] Authorization:[Digest username="icholy", realm="DLI LPC92601002528", nonce="jiviS2HZ+nV2E6yN", uri="/restapi/relay/outlets/=0,1,2/state?value=true", response="23c561a1e31a475a02ad4d49c84d840c", algorithm=MD5, cnonce="98bf1788977409f4900290f52bb4fe8a", opque="YVjDroJeBJTjdrh/", qop=auth, nc=00000001] Content-Type:[application/x-www-form-urlencoded] X-CSRF:[x]] {0xc0000aa0e0} 0x123ef80 10 [] false 192.168.0.100 map[] map[] <nil> map[] <nil> <nil> <nil> 0xc000136008}
&{401 Unauthorized 401 HTTP/1.1 1 1 map[Accept-Ranges:[dli-depth] Cache-Control:[max-age=0, private, must-revalidate] Content-Security-Policy:[frame-ancestors 'self';default-src 'self';media-src 'none';img-src 'self' data:;object-src 'none'] Content-Type:[application/json] Vary:[origin, accept, prefer] Www-Authenticate:[Digest algorithm="MD5", opaque="3yIG+X8Mi/8wOL9m", qop="auth,auth-int", realm="DLI LPC92601002528", nonce="P64iOB3eNNOHJVjS"] X-Content-Type-Options:[nosniff] X-Frame-Options:[sameorigin]] 0xc0000980c0 -1 [] true false map[] 0xc0000c4000 <nil>}
I wrote a test comparing the curl credentials against my code's credentials for the same challenge. Strange thing is, it passes.
Lines 106 to 124 in 8897bb9
The only difference I can see is the order of the parameters in the formatted version:
# curl
Digest username="icholy", realm="DLI LPC92601002528", nonce="NZAeQHhoCNifFjFa", uri="/restapi/relay/outlets/=0,1,2/state/", cnonce="MzI1MWE0MDI1MzEyOWQ2M2U1YjM1OGZiNWMwZWNiYjA=", nc=00000001, qop=auth, response="9e0d2169b41cbb504a58995e08b10eb1", opaque="wRtIEgb/X9z7XXAT", algorithm="MD5"
# icholy/digest
Digest username="icholy", realm="DLI LPC92601002528", nonce="NZAeQHhoCNifFjFa", uri="/restapi/relay/outlets/=0,1,2/state/", response="9e0d2169b41cbb504a58995e08b10eb1", algorithm=MD5, cnonce="MzI1MWE0MDI1MzEyOWQ2M2U1YjM1OGZiNWMwZWNiYjA=", opque="wRtIEgb/X9z7XXAT", qop=auth, nc=00000001
Perhaps the DLI server's digest implementation is dependent on curls specific parameter ordering. To test this, I've altered the credential formatting in the issue2 branch
Found the problem: 88bd65c I tagged a new v0.1.9 release.
I can't believe I didn't catch this problem. Thank you for the help! It is working properly now.
Please let me know if you would ever like a hand with this project!
Lol I feel the same way.