Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.

Home Page:https://llamafile.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Feature Request: Option to specify base URL for server mode

vlasky opened this issue · comments

I've been testing the use of Nginx as a proxy to serve llamafile under a subdirectory.

i.e. to be able to access the llamafile server via a URL like this:

https://mydomain.com/llamafile/

Llamafile is being launched via systemd under Linux.

I am using a rewrite rule in Nginx to remove the llamafile/ subdirectory in the URL before passing the request to llamafile.

Although I am successfully able to open the index page, it doesn't load correctly because the HTML contains Javascript code that imports the following scripts from the following absolute URLs:

/index.js
/completion.js
/json-schema-to-grammar.mjs

This is a result of the following HTML lines in the index page:

<script type="module">
    import {
      html, h, signal, effect, computed, render, useSignal, useEffect, useRef, Component
    } from '/index.js';

    import { llama } from '/completion.js';
    import { SchemaConverter } from '/json-schema-to-grammar.mjs';

At the moment, llamafile does not have a command line option to specify a base URL or URL subdirectory prefix.

The --path command line option cannot be used for this purpose.

It would be great if a base URL/URL subdirectory prefix command line option could be added to llamafile.