serverless / serverless-python-requirements

⚡️🐍📦 Serverless plugin to bundle Python packages

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Please add support for Podman OCI

schinnaswamy opened this issue · comments

Is there an existing issue for this?

  • I have searched existing issues, it hasn't been reported yet

Use case description

When we tried running the Serverless-python-requirments plugin with Podman OCI runtime we are unable to build the pythonrequirments.zip

Getting the below error when I tried running the below command

sls requirements install

Environment: darwin, node 20.5.1, framework 3.34.0 (local) 3.34.0v (global), plugin 6.2.3, SDK 4.3.2
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Error: spawn docker ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:286:19)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Tried modifying directly in the docker.js with command as podman instead of docker in

async function dockerCommand(options, pluginInstance) {
  const cmd = 'Podman';

Error stack:

Environment: darwin, node 20.5.1, framework 3.34.0 (local) 3.34.0v (global), plugin 6.2.3, SDK 4.3.2
Docs: docs.serverless.com
Support: forum.serverless.com
Bugs: github.com/serverless/serverless/issues

Error:
Running "docker run --rm -v /Users/xxx/Library/Caches/serverless-python-requirements/f2a9c040ceb6c9df0b4f83fbcb8b91d74947ceb4a90d97443c866b1c103eb1d7_arm64_slspyc:/var/task:z -v /Users/xxx/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 public.ecr.aws/sam/build-python3.9:latest-arm64 /bin/sh -c python3.9 -m pip install --platform manylinux2014_aarch64 --only-binary\=\:all\: -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && find /var/task -name \*.so -exec strip \{\} \;" failed with: ""

Proposed solution (optional)

No response

Hey @schinnaswamy 👋 Thanks for reporting and sorry you've run into trouble. Is podman CLI compatible with docker CLI? Did you try aliasing locally podman to docker? Plugin uses docker pretty transparently, so as long as the API is the same, it should work just fine, I'm not using podman personally so it's hard for me to verify at the moment.

@pgrzesik
Thanks for looking at the issue.

Yes I tried aliasing Podman to docker. But after making couple of modifications in the following .js files and removing :z from bind path syntax in the docker run command since Podman has some issues with extended attribute was getting error

lsetxattr /: operation not supported

Step 1:

alias docker=podman

Even though I have added alias but still for some reason the command was trying to connect to the docker daemon via API.

So I have to modify explicitly the following js files in the serverless-python-requirements

Step 2 : Modified docker.js Line No : 13.

const cmd = 'podman';

Step 3: Modified pip.js Line No: 269 & 371 & 416

dockerCmd.push('podman', 'run', '--rm', '-v', ${bindPath}:/var/task);

case 'podman':

if (cmd === 'podman' && e.stderrBuffer) {

Step 4: Removed :z in the pip.js file [ Line No: 269, 278, 280, 282 & 311 ]

With the above following modifications managed to run the serverless deploy with Python requirements without any issues.

Hey @schinnaswamy - thanks a lot for more details. That's interesting that the compatibility doesn't seem to be universal, at least in case of use via the plugin. I will try to find some time to run it myself via podman, but I cannot promise anything concrete right now. I would like to avoid having specific code modifications for podman if possible to reduce complexity on plugin side