No model is shown in web-ui with docker

Question

No model is shown in web-ui with docker

Cabanera opened this issue a year ago · comments

OS: 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
Docker version:

Client:
 Version:           20.10.5+dfsg1
 API version:       1.41
 Go version:        go1.15.15
 Git commit:        55c4c88
 Built:             Mon May 30 18:34:49 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.5+dfsg1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.15.15
  Git commit:       363e9a8
  Built:            Mon May 30 18:34:49 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.13~ds1
  GitCommit:        1.4.13~ds1-1~deb11u3
 runc:
  Version:          1.0.0~rc93+ds1
  GitCommit:        1.0.0~rc93+ds1-5+deb11u2
 docker-init:
  Version:          0.19.0
  GitCommit:

I've followed the installation guide for docker, and tried adding the following models in the models directory:

But in the webui no model is shown
Image webui

No error log with docker logs

Utkarsh Yadav · Answer 1 · Thu Apr 06 2023 04:00:38 GMT+0800 (China Standard Time)

what command you used to run alpaca turbo using Docker

Cabanera · Answer 2 · Thu Apr 06 2023 04:01:20 GMT+0800 (China Standard Time)

docker-compose up

Utkarsh Yadav · Answer 3 · Thu Apr 06 2023 04:03:04 GMT+0800 (China Standard Time)

send me the output of

can you get inside the container using
docker-compose exec alpaca /bin/bash

and show me the output of
ls /app/models/

Cabanera · Answer 4 · Thu Apr 06 2023 04:05:43 GMT+0800 (China Standard Time)

send me the output of

can you get inside the container using docker-compose exec alpaca /bin/bash

and show me the output of ls /app/models/

ls /app/models/
ggml-alpaca-7b-q4.bin ggml-model-q4_0.bin ggml-model-q4_1.bin

Utkarsh Yadav · Answer 5 · Thu Apr 06 2023 04:10:39 GMT+0800 (China Standard Time)

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

Cabanera · Answer 6 · Thu Apr 06 2023 04:40:31 GMT+0800 (China Standard Time)

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

i've just tried with conda, the models are showing but the error for each one is the following (paste just one of them but is the same ) :

[
'bin/main',
'-i',
'--seed',
'888777',
'-ins',
'-t',
'4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
 '--repeat_last_n',
 '64',
'--repeat_penalty',
'1',
 '--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR
[

Cabanera · Answer 7 · Thu Apr 06 2023 05:32:06 GMT+0800 (China Standard Time)

With another model

can you try running with conda or using pipenv once just want to find out whether issue is with docker or not

i've just tried with conda, the models are showing but the error for each one is the following (paste just one of them but is the same ) :
[
'bin/main',
'-i',
'--seed',
'888777',
'-ins',
'-t',
'4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
 '--repeat_last_n',
 '64',
'--repeat_penalty',
'1',
 '--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]
ERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoRERRoR
[

with another model the previous error disappears but in webui it remains stuck in loading. And the log fle is the following:

[
'bin/main',
'-i',
'--seed',
 '888777',
'-ins',
'-t',
 '4',
'-b',
'256',
'--top_k',
'1000',
'--top_p',
'0.95',
'--repeat_last_n',
'64',
'--repeat_penalty',
'1',
'--temp',
'0.2',
'--n_predict',
'200',
'-m',
'models/ggml-model-q4_0.bin',
'--interactive-first'
]

Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

That "Working" complete in 0 sec seems suspicious.

Samet Güzeldemirci · Answer 8 · Thu Apr 13 2023 22:44:52 GMT+0800 (China Standard Time)

Same issue, the model sits inside docker container but the webui is not showing it

Spiider2020 · Answer 9 · Fri Apr 14 2023 15:23:09 GMT+0800 (China Standard Time)

I had the same issue.
For me the problem was I was running Docker on my server (network machine).
The way I got it working is by getting the source code and editing it and then rebuilding.

Load the ui folder into your code editor.
Go to src/app/environments/environment.prod.ts and change the https site to http://(network machine ip):5000

Then go to angular.json find and change the config as bellow:

"configurations": {
            "production": {
              "fileReplacements": [
                {
                  "replace": "src/app/environments/environment.ts",
                  "with": "src/app/environments/environment.prod.ts"
                }
              ],
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "500kb",
                  "maximumError": "1mb"
                },
                {
                  "type": "anyComponentStyle",
                  "maximumWarning": "2kb",
                  "maximumError": "4kb"
                }
              ],
              "outputHashing": "all"
            },

Then

npm install
npm run build

Go into your docker folder where you have your project (alpaca-turbo folder), into templates , create a folder called whatever you like and move the contents of templates to it.
Then copy everything from /dist/angular-turbo where you built the angular ui to alpaca-turbo/templates .

Now you should see your .bin files

Edit:
Or even easier , you can do what this guy said
#39 (comment)