Android: Cannot load models, stopCompletions not working.

[Vali-98]

As it says on the tin. Loading small 3b models ala Tiny Llama or StableLM models do not work. Tested models:

• https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF
• https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v0.3-GGUF
• https://huggingface.co/TheBloke/phi-2-GGUF/tree/main

Attempting to call initLlama results in

• Error: Failed to initialize context

Which I can only assume is here:

• https://github.com/mybigday/llama.rn/blob/main/android/src/main/java/com/rnllama/RNLlama.java?plain=1#L55

I do not know enough about native functions to investigate further.

In addition, stopCompletions() does not stop a completion on Android.
Thanks for your work, the project is fantastic otherwise.

[Jhen-Jie Hong]

Can you share the following info? Thanks!

• llama.rn version (latest: 0.3.0-rc.9)
• Used model quantized type or file size
• Android device version & arch & total memory

Also you can use adb logcat -s "RNLLAMA_LOG_ANDROID","RNLLAMA_ANDROID_JNI" to get logs.

[Vali-98]

Sure thing:

• llama.rn version: 0.3.0-rc.9

• Models used & quantizations:

• Not working:
  -- phi-2.Q3_K_M.gguf
  -- stablelm-zephyr-3b.Q3_K_L.gguf

• Working:
  -- tinyllama-1.1b-chat-v0.3.Q3_K_L.gguf

• Android Devices Tested on:
  -- Samsung A71 - Android 13 - 8GB
  -- Emulated Pixel 3a - Android 14 - 8GB allocated

• logcat log:

RNLLAMA_ANDROID_JNI: [RNLlama] is_model_loaded false
RNLLAMA_ANDROID_JNI: [RNLlama] is_model_loaded false
RNLLAMA_ANDROID_JNI: [RNLlama] is_model_loaded false

[Vali-98]

After further testing, this seems to be caused by an older version of the package being installed instead. Purging cache and downloading latest fixed the issue.

A bit of an embarassing mistake by me, but I will leave this up for future reference. Thank you for your time.