Bad Request when sending Commit Actions that has Base64 content/encoding
kingmesal opened this issue · comments
Description
Are there tests to verify that a commit action with base64 encoded content works?
// this works
const c: CommitAction = {
action: "update",
filePath: path,
content: fromBase64(content),
};But then this fails...
// this does not work
const c: CommitAction = {
action: "update",
filePath: path,
content,
encoding: "base64",
};The string was encoded using js-base64.
GitbeakerRequestError: Bad Request
at throwFailedRequestError (file:///node_modules/.pnpm/@gitbeaker+rest@39.34.3/node_modules/@gitbeaker/rest/dist/index.mjs:41:9)
at async defaultRequestHandler (file:///node_modules/.pnpm/@gitbeaker+rest@39.34.3/node_modules/@gitbeaker/rest/dist/index.mjs:77:7)
at null.<anonymous> (async file:///apps/backend/.wrangler/tmp/dev-U35PcG/functionsWorker-0.11571656584824952.js:48661:22)
As of right now, there are none double checking the encoding. The input shouldnt be modified before sending it onto the Gitlab API, so id have to dig a bit deeper to see where the issue is. Have you tried sending the request to the API directly with those commit action options? Is the response the same (Bad Request) or does it go through fine?
I had all of my API calls to gitlab hand coded and everything worked.
When I converted this command over I copied the action, path and encoding over, 1:1 ... and it fails with gitbeaker.
I was scratching my head at this and could not figure out why, swapping base64 encoding libs and nothing worked. When I put my old code back in place it worked.
I went through gitbeaker looking for anything related to base64 and could not find anything relevant that might be a cause.
Weird, ill tinker with it over the weekend and see what I can find!
I havent gotten around to this yet, but i havent forgotten
I tried to recreate this error but came up short. This is the payload i used:
api.Commits.create('my-project', 'test', 'my next message', [
{
"action": "create",
"file_path": "foo/goo",
"content": Buffer.from("some content").toString('base64'),
"encoding": "base64"
},
]
For the base64 tests I ran I used
- atob or btoa
- js-base64
Which are the primary tools I've always used for base64 encodings.
Probably shouldn't use btoa and atob as theyve been deprecated, but just in case, I tested with both just to verify and didn't have an issue. Is there any other information you can provide? Was there anything special about the content? Do you have a snippet i could run that would recreate the problem?
So, my content was already base64 encoded. So I commented out the 2 lines at the bottom and manually decoded the content and it worked fine.
import { fromBase64 } from "js-base64";
// in my method
commits.push({
action: "update",
filePath: path,
content: fromBase64(content),
// content,
// encoding: "base64",
});Hmm it defaults to text if the encoding isnt passed.
Im not sure if its important, but when i used the js-base64 library i used the encode method like:
import { encode } from 'js-base64';
const content = 'content'
api.Commits.create('jdalrymple/test-project', 'test', 'my next message', [
{
"action": "update",
"file_path": "foo/bop",
"content": encode(content),
"encoding": "base64"
},
])
another way I had tested it was to get a full file from gitlab, which was base64 encoded and then I tried to just write that back to gitlab and it yelled at me.
Do you have an example file i use ? I want to try and recreate the behavior exactly if possible.
I tried multiple files from gitlab ... they were all just text/ html files.
And how did you download the file from gitlab? Directly? through the API, through a curl command directly? haha Does it only happen with downloaded files from gitlab? What about plain text file not download from gitlab? Do you run into the problem then as well?
It was all testing via the API .. I had my original API implementation which worked, then I just swapped gitbeaker api in ... so there was no real code change.
When I was testing it tried it with just hard coding a string and base64 encoding to send it and it yelled at me for that too.
I just went in and put the broken lines back in to run it and I get
{
cause: {
description: '13:write updated blob: writing blob: illegal base64 data at input byte 0.',
request: Request {
keepalive: false,
integrity: '',
cf: undefined,
signal: [AbortSignal],
fetcher: null,
redirect: 'follow',
headers: [Headers],
url: 'https://gitlab.com/api/v4/projects/{MYREPOID}/repository/commits',
method: 'POST',
bodyUsed: true,
body: [ReadableStream]
},
response: Response {
cf: undefined,
webSocket: null,
url: 'https://gitlab.com/api/v4/projects/{MYREPOID}/repository/commits',
redirected: false,
ok: false,
headers: [Headers],
statusText: 'Bad Request',
status: 400,
bodyUsed: true,
body: [ReadableStream]
....
}When I was testing it tried it with just hard coding a string and base64 encoding to send it and it yelled at me for that too.
Do you see a difference from when I did exactly this? Is there anything that stands out from the examples I tried that I'm missing?
Potentially just the method of encoding. The fact that mine uses content: fromBase64(content), shows that clearly it can be decoded properly...
I am not 100% that encode / decode are exported as toBase64 / fromBase64
All of our code uses toBase64 to encode what we are sending....
Im guessing you extended as per the docs?
And yea they are exported by default:
Anyways, if you can create a sample repo that recreates the issue, i can delve into the problem deeper
Gonna close this for now. If youre able to recreate in a test project ill reopen