docker run pip install ignore mismatch without any error/warning being raised

[Roy Pasternak]

After hours of trying to solve this thing - here is what I learned:

Packaging your lambda zip file using dockerizePip: true WILL NOT alert if there is a mismatch with a dependency, for example:
Ignoring boto3: markers 'python_version >= "3.9" and python_version < "4.0"' don't match your environment. (this is the output of the pip install command)
The above line will not be printed even when using SLS_DEBUG=* or --verbose and an error will not be raised.
This will result in a zip file without any dependencies (because they all were ignored).

Versions:

serverless --version
Running "serverless" from node_modules
Framework Core: 3.18.2 (local) 3.18.2 (global)
Plugin: 6.2.2
SDK: 4.3.2

"serverless-python-requirements": "^5.4.0"

Configurations:

serverless.yml

provider:
  runtime: python3.8

pythonRequirements:
    dockerizePip: true

pyproject.toml

[tool.poetry.dependencies]
python = "^3.9"
boto3 = "^1.23.6"

SLS_DEBUG=* serverless package --verbose

Logs:

Generating requirements.txt from "pyproject.toml"
Parsed requirements.txt from pyproject.toml in /path/to/file/requirements.txt
Installing requirements from "/path/to/file/requirements.txt"
Docker Image: lambci/lambda:build-python3.8
Running docker run --rm -v /path/to/requirements\:/var/task\:z -u 0 lambci/lambda\:build-python3.8 /bin/sh -c 'python3.8 -m pip install -t /var/task/ -r /var/task/requirements.txt && find /var/task -name \\*.so -exec strip \\{\\} \\;'...
✔ Service packaged (2760s)

Parsed requirements.txt from pyproject.toml:

boto3==1.23.7 ; python_version >= "3.9" and python_version < "4.0"

At first, it seems like it indeed succeeded to pack the zip file - but after running the lambda and getting import error I noticed that boto3 was missing from the actual zip file.

The expected behavior here is to at least print the fact that pip install is ignoring the boto3 dependency because of the python version mismatch:
WARNING: Ignoring boto3: markers 'python_version >= "3.9" and python_version < "4.0"' don't match your environment

I was only able to reproduce pip install output when running the docker run command on its own...

Finally, to solve the above all I had to do is to change the python version requirement in the pyproject.toml to python = "^3.8" - but this was almost impossible to detect without any printing from the pip install from inside the docker run command.