Problem of 403 between minio and ngrok with signed Url "SignatureDoesNotMatch", how to solve it ?
Aarbel opened this issue · comments
Hi minio team,
We are using ngrok to expose our server on the web from a local environnement.
Minio is accessible on "https://local.mydomain.dev/api/minio", i can connect well to the minio admin panel with my credential.
BUT
When i send a PUT request to this url with a signing url i get a 403
REQUEST
PUT /api/minio/clovis-files/18214557-486f-4cdd-a714-a585c643c8b5 HTTP/1.1
X-Forwarded-For: 92.184.99.171
X-Forwarded-Proto: https
User-Agent: aws-sdk-nodejs/2.1046.0 darwin/v16.16.0 promise
Connection: keep-alive
Expect: 100-continue
X-Amz-Date: 20240427T213815Z
Content-Length: 0
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
Authorization: AWS4-HMAC-SHA256 Credential=minio/20240427/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=c8f3df97e80876d9c1ca1fbb8973a1cc78e031366af5e9b780fd8132940571d6
X-Forwarded-Host: local.mydomain.dev
Content-Type: application/octet-stream
RESPONSE
HTTP/1.1 403 Forbidden
Date: Sat, 27 Apr 2024 21:38:15 GMT
Vary: Origin, Accept-Encoding
Content-Length: 467
X-Xss-Protection: 1; mode=block
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Content-Type: application/xml
Server: MinIO
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Amz-Request-Id: 17CA40D5E9FEE815
Content-Security-Policy: block-all-mixed-content
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><Key>18214557-486f-4cdd-a714-a585c643c8b5</Key><BucketName>clovis-files</BucketName><Resource>/clovis-files/18214557-486f-4cdd-a714-a585c643c8b5</Resource><RequestId>17CA40D5E9FEE815</RequestId><HostId>223d97b4-522d-4328-b934-c2c0d386028b</HostId></Error>
What can i do to avoid this 403 ?
Thanks a lot for your help
have you checked --host-header=rewrite in ngrok ? host is a singed header. So check the proxy/lb config.
@prakashsvmx no it didn't solve the problem, can you keep this issue open ?
REQUEST catched by ngrok
PUT /api/minio/clovis-files/3527410b-2c83-4f78-a4c5-2ed924840147 HTTP/1.1
X-Forwarded-Host: local.mydomain.dev
Connection: keep-alive
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20240429T124728Z
X-Forwarded-Proto: https
Expect: 100-continue
Authorization: AWS4-HMAC-SHA256 Credential=minio/20240429/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=606ae9ee07c0289ae96ea432d361d8983e2d9ac594c0009958c4dd268c14b1d7
User-Agent: aws-sdk-nodejs/2.1046.0 darwin/v16.16.0 promise
Content-Type: application/octet-stream
Content-Length: 12
X-Forwarded-For: 92.184.99.170
Sample17.txt
RESPONSE catched by ngrok
HTTP/1.1 403 Forbidden
X-Xss-Protection: 1; mode=block
Accept-Ranges: bytes
X-Amz-Request-Id: 17CAC10834C17E14
X-Content-Type-Options: nosniff
Access-Control-Allow-Origin: *
Server: MinIO
Content-Type: application/xml
Content-Security-Policy: block-all-mixed-content
Date: Mon, 29 Apr 2024 12:47:29 GMT
Vary: Origin, Accept-Encoding
Strict-Transport-Security: max-age=31536000; includeSubDomains
Content-Length: 467
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><Key>3527410b-2c83-4f78-a4c5-2ed924840147</Key><BucketName>clovis-files</BucketName><Resource>/clovis-files/3527410b-2c83-4f78-a4c5-2ed924840147</Resource><RequestId>17CAC10834C17E14</RequestId><HostId>91b9fc65-871d-4ef1-bf7a-dffd4af1a460</HostId></Error>
const waitPort = require('wait-port');
const ngrok = require('ngrok');
const openurl = require('openurl');
/* Wait for localhost:3000 to be ON */
const vitePort = parseInt(process.env.VITE_PORT);
waitPort({port: vitePort})
.then(() => {
ngrok.connect({
proto: 'http',
region: 'eu',
host_header: 'rewrite', // necessary to allow ngrok to connect to minio and host
addr: vitePort,
hostname: process.env.CAPACITOR_SERVER_DOMAIN,
/* Fill here the right ngrok domain if you need to work on mobile, comment this line if working on desktop */
authtoken: "XXXXXXXXXXX"
})
.then(result => {
console.info("Clovis ngrok connected", result);
/* Open the url in the default browser */
openurl.open(result);
})
.catch(console.error);
})After displaying the logs in minio i get this:
I have absolutely no clue why minio is sending a 403, even with the logs
local.mydomain.dev [REQUEST s3.CopyObject] [2024-04-30T17:31:39.927] [Client IP: 31.32.37.236]
local.mydomain.dev PUT /clovis-temporary/c59a7eb7-cc94-49c7-844b-921fa24c5ecb
local.mydomain.dev Proto: HTTP/1.1
local.mydomain.dev Host: local.mydomain.dev
local.mydomain.dev User-Agent: aws-sdk-nodejs/2.1046.0 darwin/v16.16.0 promise
local.mydomain.dev X-Amz-Date: 20240430T153139Z
local.mydomain.dev X-Forwarded-Proto: https
local.mydomain.dev Content-Length: 0
local.mydomain.dev Content-Type: application/octet-stream
local.mydomain.dev X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
local.mydomain.dev X-Forwarded-For: 31.32.37.236
local.mydomain.dev Accept-Encoding: gzip
local.mydomain.dev Authorization: AWS4-HMAC-SHA256 Credential=minio/20240430/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date, Signature=fa9bfbc24c0f7855f7b1fbaf05e6d63a6b52c84c47aa81a7138af3aa3eff284f
local.mydomain.dev Connection: close
local.mydomain.dev X-Amz-Acl: public-read
local.mydomain.dev X-Amz-Copy-Source: clovis-public-assets/project_avatar.jpg
local.mydomain.dev X-Forwarded-Host: local.mydomain.dev
local.mydomain.dev
local.mydomain.dev [RESPONSE] [2024-04-30T17:31:39.928] [ Duration 344µs TTFB 343.333µs ↑ 196 B ↓ 798 B ]
local.mydomain.dev 403 Forbidden
local.mydomain.dev Content-Type: application/xml
local.mydomain.dev Strict-Transport-Security: max-age=31536000; includeSubDomains
local.mydomain.dev X-Amz-Request-Id: 17CB189255C1FD95
local.mydomain.dev X-Xss-Protection: 1; mode=block
local.mydomain.dev Accept-Ranges: bytes
local.mydomain.dev Content-Security-Policy: block-all-mixed-content
local.mydomain.dev Server: MinIO
local.mydomain.dev Vary: Origin,Accept-Encoding
local.mydomain.dev X-Content-Type-Options: nosniff
local.mydomain.dev Content-Length: 475
local.mydomain.dev <?xml version="1.0" encoding="UTF-8"?> <Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><Key>c59a7eb7-cc94-49c7-844b-921fa24c5ecb</Key><BucketName>clovis-temporary</BucketName><Resource>/clovis-temporary/c59a7eb7-cc94-49c7-844b-921fa24c5ecb</Resource><RequestId>17CB189255C1FD95</RequestId><HostId>03ac2568-553c-4908-a89d-644926051599</HostId></Error>
local.mydomain.dev
@prakashsvmx do you have any idea of the problem ? Or how to better understand why we get a 403 ?
You can't change the port. That affects the signature.
You can't change the port. That affects the signature.
thanks @klauspost, so what is the solution to make a signed request that passes thru a proxy ?
REQUEST (url A) => proxy => local minio (url B)
I mean, what i just need is having the details of "WHY" minio throws a 403. What exactly doesn't match in the signature ?
- Is that a host origin problem ?
- Is that a timestamp problem ?
- Is that a header problem ?
other examples, with 200 and 403 for no reason
PUT /clovis-temporary/c244da58-ea28-4aff-9f24-e7bd3c75fd16
Proto: HTTP/1.1
Host: localhost:9000 //
Content-Length: 0
X-Amz-Acl: public-read // OK
X-Amz-Date: 20240430T155644Z
Authorization: AWS4-HMAC-SHA256 Credential=minio/20240430/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date, Signature=07a9c7e6bdd90b25959b83c33c0b5a607b3fa9de453e0666ab65cc14369deccb
Connection: close
Content-Type: application/octet-stream
User-Agent: aws-sdk-nodejs/2.1046.0 darwin/v16.16.0 promise
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Copy-Source: clovis-public-assets/project_avatar.jpg
[RESPONSE] [2024-04-30T17:56:44.739] [ Duration 19.171ms TTFB 19.050291ms ↑ 129 B ↓ 589 B ]
200 OK
Content-Length: 232
ETag: "06cf43f413d999212d5157a07f20ab8d"
Server: MinIO
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin,Accept-Encoding
X-Amz-Request-Id: 17CB19F0B25F2F8F
X-Content-Type-Options: nosniff
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
X-Xss-Protection: 1; mode=block
<?xml version="1.0" encoding="UTF-8"?> <CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LastModified>2024-04-30T15:56:44.721Z</LastModified><ETag>"06cf43f413d999212d5157a07f20ab8d"</ETag></CopyObjectResult>
localhost:9000
_____
PUT /clovis-temporary/3d0ad46d-805c-4b7b-b5c1-eea26521f81c HTTP/1.1
Content-Type: application/octet-stream
Connection: keep-alive
X-Forwarded-For: 31.32.37.236
Content-Length: 0
X-Amz-Copy-Source: clovis-public-assets/org_avatar.jpg
X-Amz-Date: 20240430T163738Z
X-Forwarded-Host: local.mydomain.dev
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
User-Agent: aws-sdk-nodejs/2.1046.0 darwin/v16.16.0 promise
X-Amz-Acl: public-read
Authorization: AWS4-HMAC-SHA256 Credential=minio/20240430/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date, Signature=65ac7ec5e5d63835ef98dd7ab24a96f891aa5f8de3c314d0d5cee661f3c140d6
X-Forwarded-Proto: https
HTTP/1.1 200 OK
Content-Type: application/xml
X-Amz-Request-Id: 17CB1C2BEE3D2908
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
Content-Security-Policy: block-all-mixed-content
Server: MinIO
Date: Tue, 30 Apr 2024 16:37:38 GMT
Accept-Ranges: bytes
Content-Length: 232
Vary: Origin,Accept-Encoding
X-Xss-Protection: 1; mode=block
Etag: "6b1500d57ca413d65f42f623d268a82e"
<?xml version="1.0" encoding="UTF-8"?>
<CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LastModified>2024-04-30T16:37:38.151Z</LastModified><ETag>"6b1500d57ca413d65f42f623d268a82e"</ETag></CopyObjectResult>
_____
PUT /api/minio/clovis-temporary/ba583648-e945-4d28-8ae5-3c545542736d HTTP/1.1
Connection: keep-alive
User-Agent: aws-sdk-nodejs/2.1046.0 darwin/v16.16.0 promise
X-Forwarded-For: 31.32.37.236
X-Amz-Copy-Source: clovis-public-assets/project_avatar.jpg
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Acl: public-read
X-Amz-Date: 20240430T165602Z
Content-Type: application/octet-stream
X-Forwarded-Host: local.mydomain.dev
Authorization: AWS4-HMAC-SHA256 Credential=minio/20240430/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date, Signature=f89a79c52cc4554ad09cc8c3b0a45a1767bc4f229141b3a47c90c433a06161af
X-Forwarded-Proto: https
Content-Length: 0
HTTP/1.1 403 Forbidden
Content-Type: application/xml
Content-Length: 475
Date: Tue, 30 Apr 2024 16:56:02 GMT
Access-Control-Allow-Origin: *
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
X-Amz-Request-Id: 17CB1D2D1D58CC7B
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
Server: MinIO
Vary: Origin, Accept-Encoding
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><Key>ba583648-e945-4d28-8ae5-3c545542736d</Key><BucketName>clovis-temporary</BucketName><Resource>/clovis-temporary/ba583648-e945-4d28-8ae5-3c545542736d</Resource><RequestId>17CB1D2D1D58CC7B</RequestId><HostId>03ac2568-553c-4908-a89d-644926051599</HostId></Error>
you preserve port and hostname @Aarbel use normal nginx etc instead of ngrok tunneling.
you preserve port and hostname @Aarbel use normal nginx etc instead of ngrok tunneling.
@harshavardhana
Indeed things work well with ngrok, my problem is when the request passes thru Vite.js proxy (exposing localhost:3000 thru thru ngrok), but i don't have any detailed error from Minio about this signing 403. So that's really really obscure with we get a 403 after passing thru Vite proxy.
const config: ViteConfig = {
...
server: {
...
proxy: {
"/api/minio": {
changeOrigin: true,
configure: (proxy, _options) => {
proxy.on("error", (err, _req, _res) => {
console.log("🔥 proxy error", err);
});
proxy.on("proxyReq", (proxyReq, req, _res) => {
// Set headers on proxyReq to match incoming request headers
// Object.keys(req.headers).forEach((header) => {
// proxyReq.setHeader(header, req.headers[header]);
// });
console.log("🔥 Sending Request method:", req.method);
console.log("🔥 Sending Request url:", req.url);
console.log("🔥 Sending Request headers:", req.headers);
console.log("🔥 Target method:", proxyReq.method);
console.log("🔥 Target protocol:", proxyReq.protocol);
console.log("🔥 Target host:", proxyReq.host);
console.log("🔥 Target path:", proxyReq.path);
console.log("🔥 Target headers:", proxyReq.getHeaders());
});
proxy.on("proxyRes", (proxyRes, req, _res) => {
console.log(
"🔥 Received Response from the Target 2:",
proxyRes.statusCode,
req.method,
req.headers,
req.url,
proxyRes.headers
);
});
},
rewrite: (path) => {
const rewrittenPath = path.replace(/^\/api\/minio/, "/");
console.log("🔥Original path:", path);
console.log("🔥Rewritten path:", rewrittenPath);
return rewrittenPath;
},
target: "http://localhost:9000",
},
},
},
};
rewrite: (path) => {
const rewrittenPath = path.replace(/^/api/minio/, "/");
console.log("🔥Original path:", path);
console.log("🔥Rewritten path:", rewrittenPath);
return rewrittenPath;
},
you can never do path rewrites they are also part of signature mechanism.
rewrite: (path) => {
const rewrittenPath = path.replace(/^/api/minio/, "/");
console.log("🔥Original path:", path);
console.log("🔥Rewritten path:", rewrittenPath);
return rewrittenPath;
},you can never do path rewrites they are also part of signature mechanism.
@harshavardhana
How can you use a proxy without path rewrite ? And is that possible to consider this path rewrite in the signature ?
For information, for now i solved this problem with using two ngrok domains, because it looks like
- minio is blocking signedUrls which path has been rewrited by a proxy
- vite.js doesn't seem to have any options to avoid this in their proxy