DavidTPate / text-generation-webui-cli

Text Generation Web UI CLI Tool

Basic CLI tool for using the Text Generation Web UI websocket API.

Getting Started

1. Clone this Repository.
2. Run npm i to install project dependencies.
3. Run npm run build to build the project.
4. Run the script locally like node ./dist/index.js --prompt "whatever"

CLI Args

Flag	Default	Description
--prompt PROMPT		Pass the desired prompt Required
--model MODEL		Change to the specified model prior to passing the prompt
--baseUrl BASE_URL	ws://127.0.0.1:7860/queue/join	the websocket "join" URL
--softPromptPath SOFT_PROMPT_PATH		Loads a template to wrap the prompt, must contain __prompt__ which will be replaced with the prompt value
--promptReplacePath PROMPT_REPLACE_PATH		Loads and parses a newline-limited list of values to generate output for each value. The prompt must contain the string __replace__ which will be (logically) replaced with a value in the list for each generation
--max_new_tokens MAX_NEW_TOKENS	200	See: HuggingFace Generation Config
--seed SEED	-1	See: HuggingFace Generation Config
--temperature TEMPERATURE	0.7	See: HuggingFace Generation Config
--top-p TOP_P	40	See: HuggingFace Generation Config
--top-k TOP_K	0.1	See: HuggingFace Generation Config
--typical-p TYPICAL_P	1	See: HuggingFace Generation Config
--repetition_penalty REPETITION_PENALTY	1.18	See: HuggingFace Generation Config
--encoder_repetitition_penalty ENCODER_REPETITION_PENALTY	1	See: HuggingFace Generation Config
--no_repeat_ngram_size NO_REPEAT_NGRAM_SIZE	1	See: HuggingFace Generation Config
--min_length MIN_LENGTH	0	See: HuggingFace Generation Config
`--do_sample true	false`	true
--penalty_alpha PENALTY_ALPHA	0	See: HuggingFace Generation Config
--num_beams NUM_BEAMS	1	See: HuggingFace Generation Config
--length_penalty LENGTH_PENALTY	1	See: HuggingFace Generation Config
`--early_stopping true	false`	false
`--add_bos_token true	false`	true
`--ban_bos_token true	false`	false
--truncation_length TRUNCATION_LENGTH	2048	See: HuggingFace Generation Config
--custom_stopping_strings CUSTOM_STOPPING_STRINGS	""	See: HuggingFace Generation Config
--gpu_memory_0 0...MAX_VRAM	0	See: HuggingFace Generation Config
--cpu_memory 0...MAX_CPU	""	See: HuggingFace Generation Config
`--auto_devices true	false`	""
`--disk true	false`	""
`--cpu true	false`	""
`--bf16 true	false`	""
`--load_in_8bit true	false`	""
`--wbits 1	2	3
`--groupsize 32	64	128`
`--model_type None	llama	opt
--pre_layer 0...100	""	See: HuggingFace Generation Config