🐛 Bug Report: Unable to createFile that is > 10MB in filesize

[mcchin]

👟 Reproduction steps

Prepare a file that is > 10MB (The one I am using is 19MB),
copy and paste the createFile example codes from Appwrite https://appwrite.io/docs/server/storage?sdk=nodejs-default#storageCreateFile, then modified the file path to your > 10MB file

storage.createFile(
    'unique()',
    fs.createReadStream(`./large_file.pdf`)
)

👍 Expected behavior

I should be able to createFile without errors

👎 Actual Behavior

It throws the following error

(node:4348) UnhandledPromiseRejectionWarning: Error: Request body larger than maxBodyLength limit
    at Client.call (/mnt/c/Users/fores/node_modules/node-appwrite/lib/client.js:169:23)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Storage.createFile (/mnt/c/Users/fores/node_modules/node-appwrite/lib/services/storage.js:97:16)
    at async run (/mnt/c/Users/fores/Downloads/latte/convert.js:124:20)

Note that no errors are thrown from Appwrite server, the above is from node sdk client

🎲 Appwrite version

Different version (specify in environment)

💻 Operating system

Linux

🧱 Your Environment

• Appwrite version 0.12.1
• node-appwrite version 4.0.2
• Setup Appwrite server environment variable _APP_STORAGE_LIMIT=60000000 (10MB to 60MB or more)

👀 Have you spent some time to check if this issue has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

[mcchin]

To make it work I have to modified lib/client.js like below:

        let options = {
            method: method.toUpperCase(),
            url: this.endpoint + path,
            params: (method.toUpperCase() === 'GET') ? params : {},
            headers: headers,
            data: (method.toUpperCase() === 'GET' || contentType.startsWith('multipart/form-data')) ? formData : params,
            json: (contentType.startsWith('application/json')),
            maxContentLength: Infinity,
            maxBodyLength: Infinity,
            responseType: responseType
        };

Basically adding the following

            maxContentLength: Infinity,
            maxBodyLength: Infinity,

to axios config

I went through the documentations and source codes I could not find out how I could override it from SDK

[Eldad A. Fux]

@mcchin thanks for reporting the issue. Appwrite 0.13 is planned to introduce big files support with range headers, streaming and chunked uploads. This will include support in all of the SDKs.

[Damodar Lohani]

@mcchin This is not a bug, you can change the _APP_STORAGE_LIMIT environment variable to increase the payload size. But, yes as @eldadfux mentioned we are working on updating this so, we can handle much larger file efficiently.

[Damodar Lohani]

Closing this issue, if the _APP_STORAGE_LIMIT variable doesn't help you resolve for now, feel free to reopen the issue.

[mcchin]

@lohanidamodar As described in my issue, I have already changed _APP_STORAGE_LIMIT_ to more than 10MB. The error is thrown by the node sdk, and as described in my issue that specifically is the Axios config.

So please reopen this ticket, so it can be tracked :)

[mcchin]

@eldadfux @lohanidamodar

I have no permission to reopen this ticket, as mentioned above _APP_STORAGE_LIMIT_ doesn't solved the problem as this is coming from the Axios client configuration on node-sdk. I hate to see this issue is missed out on next release, can someone reopen it?

Thanks

[Damodar Lohani]

@mcchin sorry for the misunderstanding. Yes you are correct, and as @eldadfux mentioned, this will be handled in new release by chunking the large file, so should be resolved in the upcoming version. Keeping the issue open until the release is out. 🙏🏻

[mcchin]

Nice, I haven't upgraded to 0.13 (appwrite), and 5.0.0 (node-sdk) respectively to test it yet,
but seeing the code changes that the file is now divided by 5MB chunk size, don't think axios will throw the large file error now

[Damodar Lohani]

Closing this as resolved in 0.13.0. If the issue persist, feel free to re-open the issue.