su77ungr / CASALIOY

♾️ toolkit for air-gapped LLMs on consumer-grade hardware

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reduce docker :stable image size

su77ungr opened this issue · comments

Problem:

  • Image size od stable >7GB
  • risking connection timeouts on low bandwidth users

I'm not sure I understand exactly what the issue is ?

7gb is too much, but a part is probably due to cuda. We can add it as optional in the poetry file.

From https://github.com/hippalectryon-0/CASALIOY/tree/better-docker

user0@DESKTOP-Q3PBFPO:~/CASALIOY$ docker image history casalioy:latest 
IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
fda593445e09   42 seconds ago   /bin/sh -c #(nop) COPY file:648de41f54afe6f9…   613B             
f67cf731fe98   42 seconds ago   /bin/sh -c pip3 uninstall -y streamlit sente…   72.2MB           
579591ecff1c   54 seconds ago   /bin/sh -c pip3 install torch torchvision --…   925MB            
b2b90d314668   2 minutes ago    /bin/sh -c python3 -m poetry install            4.21GB                          
045d81b418d8   4 minutes ago    /bin/sh -c python3 -m poetry config virtuale…   248kB                           
23f73c9e9eaa   4 minutes ago    /bin/sh -c pip3 install poetry                  100MB                           
7c6a0ec932e0   4 minutes ago    /bin/sh -c git checkout better-docker           4.23kB                          
7f40ef97490c   5 minutes ago    /bin/sh -c #(nop) WORKDIR /srv/CASALIOY         0B

REPOSITORY TAG IMAGE ID CREATED SIZE
casalioy latest 46a3bc99c3cd 5 minutes ago 2.57GB

We're getting there !

sub 3 sounds reasonable. nice