Installing psycopg2-binary with Python:3.6.4-alpine doesn't work
barrachri opened this issue · comments
When you try to install the binary version of the lib you get the source version error.
$ docker run -it --rm python:3.6.4-alpine sh
$ pip install psycopg2-binary
Collecting psycopg2-binary
Downloading psycopg2-binary-2.7.4.tar.gz (426kB)
100% |████████████████████████████████| 430kB 52kB/s
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2_binary.egg-info
writing pip-egg-info/psycopg2_binary.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/psycopg2_binary.egg-info/dependency_links.txt
writing top-level names to pip-egg-info/psycopg2_binary.egg-info/top_level.txt
writing manifest file 'pip-egg-info/psycopg2_binary.egg-info/SOURCES.txt'
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.
For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
Alpine Linux use musl libc and manylinux wheels don't support it: see e.g. pypa/manylinux#37. Not a psycopg problem but a lack of a standard for it, such as PEP 513 defininig manylinux1. Happy to support Alpine as soon as there will be a standard and tools for it.
I found this solution which worked quite fine.
FROM python:3.6-alpine
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
pip install psycopg2-binary
Edit
The above compiles the package from source, so I would rather
FROM python:3.6-alpine
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
pip install psycopg2
not to get confused.
But I think you are pseudo-using the binary package and building from the source at the end.
@nourchawich your "solution" is about compiling psycopg from source, which is totally supported. Binary packages are not (for the reason described above).
Thank for the clarification @barrachri and @dvarrazzo
I just noticed what was actually happening. All I wanted was to have psycopg2 (regardless whether binary or not) installed on an alpine based image, hence considering it a solution.
@nourchawich after it's been built, I suppose we can get rid of those dev packages ?
@nourchawich after it's been built, I suppose we can get rid of those dev packages ?
Yes, but make sure the apt add and apt del commands are in one layer. Otherwise you won't make the image any smaller.
E.g.:
RUN apk add --no-cache --virtual .build-deps \
gcc \
python3-dev \
musl-dev \
postgresql-dev \
&& pip install --no-cache-dir psycopg2 \
&& apk del --no-cache .build-depsYeah I did, thanks !
Since this is the top google hit for this problem, use a multi-stage build to minimize the final image size.
FROM python:3.6-alpine as base
FROM base as builder
RUN mkdir /install
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
WORKDIR /install
COPY requirements.txt /requirements.txt
RUN pip install --install-option="--prefix=/install" -r /requirements.txt
FROM base
COPY --from=builder /install /usr/local
COPY src /app
RUN apk --no-cache add libpq
WORKDIR /app
This is still a bit unclear to me. The point of installing psycopg2-binary is that compilers like gcc etc. are not necessary right?
I'm getting the same message as the original poster. When trying to install psycopg2-binary, it complains about not finding gcc and suggests to, well, install psycopg2-binary (which is what I'm trying to do).
I'm on Centos 7.4, and apparently I had to yum install postgresql-devel to make psycopg2-binary to work.
@hugobuddel The problem is when pip decides it can't install the psycopg2-binary wheel it falls back on building from source, which naturally requires dev packages. On alpine that's because of muslc instead of glibc. I should think this shouldn't happen on CentOS but maybe there's another conflict.
Building on @Cerebus's answer, to get this to work with pipenv, you can try something like this:
FROM python:3.6-alpine as base
FROM base as builder
RUN mkdir /install
RUN apk update \
&& apk add postgresql-dev gcc python3-dev musl-dev
RUN pip install pipenv # to lock and create requirements.txt
WORKDIR /install
# creation of requirements.txt
COPY Pipfile ./Pipfile
RUN pipenv lock -r > requirements.txt
# everything else stays the same
RUN pip install --install-option="--prefix=/install" -r ./requirements.txt
FROM base
COPY --from=builder /install /usr/local
COPY . /app
RUN apk --no-cache add libpq
WORKDIR /app
While it is possible to use multi-stage builds or add libraries to compile from source, I recently started using the slim version instead of alpine.
It installs psycopg2-binary without issues.
FROM python:3.6-slim
pip install psycopg2-binary
I found this article to be very useful in deciding which Python base image to use for production.
https://pythonspeed.com/articles/base-image-python-docker-images/
@MosheVai pipenv appears to have pulled a Lazarus as of a few hours ago.
Alpine Linux use musl libc and manylinux wheels don't support it: see e.g. pypa/manylinux#37. Not a psycopg problem but a lack of a standard for it, such as PEP 513 defininig
manylinux1. Happy to support Alpine as soon as there will be a standard and tools for it.
Looks like musllinux support has been added to manylinux: pypa/manylinux#1135
Perhaps this issue could have a revisit?
FROM python:3.10.4-alpine
RUN apk add --no-cache --virtual build-dependencies libpq-dev build-base \
&& pip install -r requirements.txt \
&& apk del --no-cache build-dependencies
seems to work too.
not able to install psycopg2 on my docker environment
requirement.txt
Django>=3.2.4,<3.3 djangorestframework>=3.12.4,<3.13 psycopg2>=2.8.6,<2.9
dockerfile
`
FROM python:3.9-alpine3.13
LABEL maintainer="*******************"
ENV PYTHONUNBUFFERED 1
COPY ./requirements.txt /tmp/requirements.txt
COPY ./requirements.dev.txt /tmp/requirements.dev.txt
COPY ./app /app
WORKDIR /app
EXPOSE 8000
ARG DEV=false
RUN python -m venv /py &&
/py/bin/pip install --upgrade pip &&
apk add --update --no-cache postgresql-client &&
apk add --update --no-cache --virtual .tmp-build-deps
build-base postgresql-dev gcc python3-dev musl-dev &&
/py/bin/pip install -r /tmp/requirements.txt &&
if [ $DEV = "true" ];
then /py/bin/pip install -r /tmp/requirements.dev.txt ;
fi && \
rm -rf /tmp &&
apk del .tmp-build-deps &&
adduser
--disabled-password
--no-create-home
django-user
ENV PATH="/py/bin:$PATH"
USER django-user
`