POST with multiple files with the form-data sends only 1 file.

[GeekBeard]

POSTing multiple files with a form-data. Using this doc: https://requests.readthedocs.io/en/latest/user/advanced/#post-multiple-multipart-encoded-files

However only 1 file is received by the endpoint.

But if you change the files array (deviates from the documentation):

from:

files=[
  ('images',('image01.png',open('H:/image01.png','rb'),'image/png')),
  ('images',('image02.png',open('H:/image02.png','rb'),'image/png'))
]

to:

files=[
  ('images01',('image01.png',open('H:/image01.png','rb'),'image/png')),
  ('images02',('image02.png',open('H:/image02.png','rb'),'image/png'))
]

everything works.

Expected Result

Receiving endpoint gets an array with 2 items to iterate through them and save them as files.

Actual Result

Receiving endpoint gets 1 item instead of 2 (or more if you are posting more).

Reproduction Steps

Test script. For the endpoint you can run something of your own or even use https://webhook.site/

import requests
url = "https://endpoint.com/api/upload"
payload = {}
files=[
  ('images',('image01.png',open('H:/image01.png','rb'),'image/png')),
  ('images',('image02.png',open('H:/image02.png','rb'),'image/png'))
]
headers = {}
response = requests.post(url, files=files)
print(response.text)

Endpoint will get only 1 file. If you change files array to make images have separate value for each file = endpoint gets 2 files.

System Information

$ python -m requests.help

{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.11"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "1010114f"
  },
  "urllib3": {
    "version": "1.26.9"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

[Kushagra Mittal]

Hello @geekbeard,

As request.post takes files as list in that also, this list should be [(key,value)] or [{key,value}] which is checked by function src.requests.utils.to_key_val_list() and which make sense that there should be unique key. But, if it is working same key in older versions then it is bug in new one and it should be fixed.

For library perspective, we can create unique field name like this images_image01 or update readme document.

I had tried with get request with same key but different values but it takes only last key and value. That's a reason this key should be unique in form-data also.

[Ian Stapleton Cordasco]

@geekbeard I believe you're correct that this is a bug. I think somewhere things are being round tripped through a dictionary and that's why you're loosing a file here.

An interim fix is to use the MultipartEncoder from the toolbelt. It doesn't have this bug

[Ashwani]

I believe the bug is on the client side.

The request seems to be sent correctly (although https://webhook.site/ only shows one file being received).
After creating my own endpoint using fastapi, I was able to see both files I uploaded.

Server:

from fastapi import FastAPI, File, UploadFile
from typing import List

app = FastAPI()

@app.post("/upload")
async def upload_files(images: List[UploadFile]):
  uploaded_files_info = []
  for file in images:
    content = await file.read()
    uploaded_files_info.append({
      "filename": file.filename,
      "content_type": file.content_type,
      "content_length": len(content)
    })
  return {"uploaded_files": uploaded_files_info}

if __name__ == "__main__":
  import uvicorn
  uvicorn.run(app, host="0.0.0.0", port=8080)

Request:

import requests
url = "http://0.0.0.0:8080/upload"

files=[
  ('images',('a.jpeg',open('/home/ash/Pictures/a.jpeg','rb'),'image/jpeg')),
  ('images',('b.jpeg',open('/home/ash/Pictures/b.jpeg','rb'),'image/jpeg'))
]

response = requests.post(url, files=files)
response.text

Response:

'{"uploaded_files":[{"filename":"a.jpeg","content_type":"image/jpeg","content_length":8155},{"filename":"b.jpeg","content_type":"image/jpeg","content_length":7104}]}'

[Ian Stapleton Cordasco]

I believe the bug is on the client side.

I believe you mean server side.

The request seems to be sent correctly (although https://webhook.site/ only shows one file being received).
After creating my own endpoint using fastapi, I was able to see both files I uploaded.

Server:

from fastapi import FastAPI, File, UploadFile
from typing import List

app = FastAPI()

@app.post("/upload")
async def upload_files(images: List[UploadFile]):
  uploaded_files_info = []
  for file in images:
    content = await file.read()
    uploaded_files_info.append({
      "filename": file.filename,
      "content_type": file.content_type,
      "content_length": len(content)
    })
  return {"uploaded_files": uploaded_files_info}

if __name__ == "__main__":
  import uvicorn
  uvicorn.run(app, host="0.0.0.0", port=8080)

Request:

import requests
url = "http://0.0.0.0:8080/upload"

files=[
  ('images',('a.jpeg',open('/home/ash/Pictures/a.jpeg','rb'),'image/jpeg')),
  ('images',('b.jpeg',open('/home/ash/Pictures/b.jpeg','rb'),'image/jpeg'))
]

response = requests.post(url, files=files)
response.text

Response:

'{"uploaded_files":[{"filename":"a.jpeg","content_type":"image/jpeg","content_length":8155},{"filename":"b.jpeg","content_type":"image/jpeg","content_length":7104}]}'

As a result I'm going to close this