Error Initializing Llama Model: Context Limit Reached

[samtin0x]

Description

When using this library on an Expo v50 app I got a persistent error that stopped the completions from working. The app was functioning correctly, but after a series of prompts, it failed with the error "[Error: Context limit reached]". I smell this is an easy fix from my end? Any help is appreciated :)

Steps to Reproduce

1. Initialize the Llama model with standard settings.
2. Send multiple messages to the model.
3. Observe the failure after several interactions.

Environment

• Expo version: v50
• Model: phi2 3B Q2_K - Medium
• Device: iPhone 14 Pro Max (development build)

Code Snippet

export const runLlama = async (message: string, modelId: string) => {
  const MODEL_FILE_PATH = `${FileSystem.documentDirectory}${modelId}.gguf`;
  return new Promise((resolve, reject) => {
    initLlama({
      model: MODEL_FILE_PATH,
      use_mlock: true,
      n_ctx: 2048,
      n_gpu_layers: 1,
    })
    .then((context) => {
      context.completion({
        prompt: message,
        n_predict: 60,
        temperature: 0.7,
        top_p: 1.0,
        stop: ['</s>', 'Llama:', 'User:'],
      }, (data) => {
        const { token } = data;
        resolve({ type: 'partial', token });
      })
      .then((sss) => {
        console.log('Completion result:', sss.text);
        resolve({ type: 'final', text: sss.text });
      })
      .catch((error) => {
        console.log('Error running Llama:', error);
        reject(error);
      });
    })
    .catch((error) => {
      console.log('Error initializing Llama:', error);
      reject(error);
    });
  });
};

Error

 ERROR  Error running Llama: [Error: Context limit reached]

[Jhen-Jie Hong]

There is the code of context limit in this module:

llama.rn/ios/RNLlama.mm

Lines 36 to 39 in 7fbcebc

 if (llamaContextLimit > 0 && [llamaContexts count] >= llamaContextLimit) { 
     reject(@"llama_error", @"Context limit reached", nil); 
     return; 
 } 

If you want to run more contexts, you can set more context limit by setContextLimit. Otherwise, you should release the existing context before initialize a new one.