Restrict access to the frontend

Question

Restrict access to the frontend

Seluj78 opened this issue a year ago · comments

Hi,

I've just setup my own pypiserver and it works great, asking for my .htpasswd user and password when downloading or uploading through pip and twine. The only problem is, the frontend where it lists the available packages isn't secured. I can just click on the links and download the packages. Have I miss configured something on my nginx.conf ?

Thanks

Jules Lasne · Answer 1 · Thu Feb 16 2023 20:11:51 GMT+0800 (China Standard Time)

Here is my nginx config. I just need pypiserver to also ask for a username/password when trying to view from a webbrowser the page packages.xxx.com.

events {}

http{

    #recommended log format
    log_format nginx '\$remote_addr - \$remote_user [\$time_local] '
                  '"\$request" \$status \$body_bytes_sent \$request_time '
                  '"\$http_referer" "\$http_user_agent"';

    access_log /var/log/nginx/access.log;

    upstream gunicorn_service {
        # server unix:/home/ubuntu/admin-platform/server.sock;
        server 127.0.0.1:8080;
    }

    server {
        listen 80;
        server_name packages.xxx.com;
        return 301 https://$server_name$request_uri;
    }

    server {
        listen 443 ssl;
        server_name packages.xxx.com;
        ssl_certificate /etc/letsencrypt/live/packages.xxx.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/packages.xxx.com/privkey.pem;
        ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers HIGH:!aNULL:!MD5;
        client_max_body_size 0;
        error_page 503 = @no_user;
        resolver 8.8.8.8;
        resolver_timeout 10s;

        location / {
            proxy_http_version 1.1;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $host;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_read_timeout 86400;
            proxy_cache off;
            proxy_pass http://gunicorn_service;
        }

    }

    include /etc/nginx/conf.d/*.conf;
}

Mitja O · Answer 2 · Fri Feb 17 2023 01:14:31 GMT+0800 (China Standard Time)

Hello @Seluj78! Thanks a lot for opening this issue! Sorry to hear that it has caused you some inconvenience, I am going to properly look into it shortly and hope to find a good solution!

In the meantime, could you share the arguments you pass to the pypiserver executable when starting it? On first look, I believe that passing --authenticate 'download, update, list' might be able to solve the problem you are describing. There are more details about the authentication argument available with pypiserver run --help. Could you give it a go? :) I will also try to reproduce this behaviour locally as soon as possible.

Mitja O · Answer 3 · Fri Feb 17 2023 01:20:34 GMT+0800 (China Standard Time)

UPD: I have just not tested this locally by running it in Docker with the --authenicate=list (make sure you specify all the required actions for your use case) parameter and it appears to work for me as expected - the access to the /packages and /simple is protected.

See the example below:

$ docker run --rm pypiserver/pypiserver run --help
usage: pypi-server run [-h] [-v] [--log-file FILE] [--log-stream STREAM] ...
   ...
  -a AUTHENTICATE, --authenticate AUTHENTICATE
                        Comma-separated list of (case-insensitive) actions to
                        authenticate (options: download, list, update;
                        default: update).
                         
                         Any actions not specified are not authenticated, so
                         to authenticate downloads and updates, but allow
                         unauthenticated viewing of the package list, you would
                         use:
                         
                          pypi-server -a 'download, update' -P
                          ./my_passwords.htaccess
                         
                        To disable authentication, use:
                         
                          pypi-server -a . -P .
                         
                        See the `-P` option for configuring users and
                        passwords.
                         
                        Note that when uploads are not protected, the
                        `register` command is not necessary, but `~/.pypirc`
                        still needs username and password fields, even if
                        bogus.
  -P PASSWORD_FILE, --passwords PASSWORD_FILE
                        Use an apache htpasswd file PASSWORD_FILE to set
                        usernames and passwords for authentication.
                         
                        To allow unauthorized access, use:
                         
                          pypi-server -a . -P .
                         
  ...
$ docker run -p 80:8080 \
   -v <your local password file>:/data/.htpasswd \
   pypiserver/pypiserver \
   run --authenticate=list \
   --password /data/.htpasswd

This is how the frontend appears when the pypiserver is started as above (the cancelled browser basic auth prompt is not recorded on the GIF):

Let me know if this helps ✌️

Jules Lasne · Answer 4 · Fri Feb 17 2023 01:24:36 GMT+0800 (China Standard Time)

Hi ! I am running pypiserver on systemd with the download, list and upload protected (on my phone at the moment, so I cannot send you the file).

Also, /packages, /simple and / are not protected in any way. If you have an email address, I can send you the real URL for you to try :)

Mitja O · Answer 5 · Fri Feb 17 2023 01:28:47 GMT+0800 (China Standard Time)

Hi ! I am running pypiserver on systemd with the download, list and upload protected (on my phone at the moment, so I cannot send you the file).

Also, /packages, /simple and / are not protected in any way. If you have an email address, I can send you the real URL for you to try :)

Hey @Seluj78, right! No worries, I think I understand the issue without the link 🙂 I made a quick attempt in the comment above, when you get the time could you give that approach a try? :)

Jules Lasne · Answer 6 · Fri Feb 17 2023 01:31:19 GMT+0800 (China Standard Time)

I'll test it out in 5 minutes and let you know !

Jules Lasne · Answer 7 · Fri Feb 17 2023 06:24:30 GMT+0800 (China Standard Time)

Sorry, might've forgotten and went out with friends ! I just verified and here's my service file (that doesn't restrict listing as it should)

[Unit]
Description=A minimal PyPI server for use with pip/easy_install.
After=network.target

[Service]
Type=simple
# systemd requires absolute path here too.
PIDFile=/var/run/pypiserver.pid

ExecStart=/home/ubuntu/venv/bin/pypi-server run -p 8080 -a update,download,list --log-file /var/log/pypiserver.log -P /etc/nginx/.htpasswd /var/www/pypi
ExecStop=/bin/kill -TERM $MAINPID
ExecReload=/bin/kill -HUP $MAINPID
Restart=always

WorkingDirectory=/var/www/pypi

TimeoutStartSec=3
RestartSec=5

[Install]
WantedBy=multi-user.target

Mitja O · Answer 8 · Sat Feb 18 2023 23:08:34 GMT+0800 (China Standard Time)

No worries @Seluj78! Thanks for sharing your service setup, and unfortunate that it didn't resolve the problem! I'll replicate the setup here and try to debug the issue in more detail. Hopefully I'll get some time for this during next week and I'll keep you posted. ✌️ If you find out anything more in the meantime, feel free to comment here, that'd help investigating :) I hope we'll figure it out soon 👍🍀

Jules Lasne · Answer 9 · Sun Feb 19 2023 20:25:02 GMT+0800 (China Standard Time)

Thank you ! :D Do let me know if you need more details on my config :)

Jules Lasne · Answer 10 · Wed Feb 22 2023 16:55:04 GMT+0800 (China Standard Time)

Hi @dee-me-tree-or-love :) Have you had a chance to look at the problem ? :)

Mitja O · Answer 11 · Wed Feb 22 2023 17:26:34 GMT+0800 (China Standard Time)

Hello @Seluj78! I'm sorry bot not, I've been a little under the load of my other work these days and didn't get to it yet. :C But I hope I can take a closer look by the end of the week ✌️ sorry for the waiting, hope that still works?

In the meantime, if it's not too much work for you, it would really help me out if you're up to set up a small replication repo which I can checkout and run locally - that'd make things real nice :) But only if you have the time of course, no worries otherwise 😃

Thanks for the ping ✌️

Jules Lasne · Answer 12 · Wed Feb 22 2023 17:28:18 GMT+0800 (China Standard Time)

Yes no worries it's fine ! :) Sadly open source cannot be the priority for many of us :/ I wish it could be the case for me :)

Of course ! What would you need in this repository ? :)

Mitja O · Answer 13 · Sun Feb 26 2023 21:41:47 GMT+0800 (China Standard Time)

@Seluj78

Yes no worries it's fine ! :) Sadly open source cannot be the priority for many of us :/ I wish it could be the case for me :)

Thanks a lot, I fully agree :) Luckily sometimes there is the time for it too still :)

Of course ! What would you need in this repository ? :)

I am now just trying to replicate your setup, but, currently, it still ends up in the front end being secured with the basic-auth prompt. So maybe if you could make a little VM setup or otherwise include all the required files to replicate your setup on a, say, Ubuntu machine, that'd be great! So the .service file the nginx.config at least and a small explanation of how to start the setup. Then I could jump into it directly. If you could also provide a GIF/a few screenshots/output copies similar to the one I attached in a comment above, that'd help as well!

Besides, there's one more question I forgot to ask, which pypiserver version and which OS (+ version) are you currently running? 😅

Thanks a lot in any case! 🙌 I'm going to give it a few more tries myself and will share the findings.

Mitja O · Answer 14 · Mon Feb 27 2023 19:33:53 GMT+0800 (China Standard Time)

Hey @Seluj78, I've created a small replica repository where I tried to reproduce your bug running a simplified setup using 1) Docker compose, 2) Vagrant: https://github.com/dee-me-tree-or-love/pypiserver-476-replica. In both cases, I managed to get the front end protected 🤔. Could you take a look there if you can spot what could be the issues? Feel free to clone it and open new detailed discussions there if you'd like!

Jules Lasne · Answer 15 · Mon Feb 27 2023 20:11:42 GMT+0800 (China Standard Time)

Thank you !

I've made an issue here: dee-me-tree-or-love/pypiserver-476-replica#1

Jules Lasne · Answer 16 · Wed Mar 29 2023 05:44:36 GMT+0800 (China Standard Time)

Closing as completed for now. One of my coworkers went on our domain name where pypiserver was hosted and had to log in to access the simple and index routes.