Android DeepSpeech loadLibrary fails for Split APKs / Dynamic Feature

[Erik Ziegler]

Hey there!

When using a Dynamic Feature Module in Android that has DeepSpeech as a dependency, calling any functionality fails because the native libs can not be found. This is because Dynamic Features result in splitted APKs making the native libraries not accessible by System.loadLibrary(...) anymore. Instead, you would need to call SplitInstallHelper.loadLibrary(...) (see here and here). But I can see that this is not a desired change because we can't know in advance if the library is used inside a Dynamic Feature.

This seems to be a general problem with using dependencies in Dynamic Feature Modules that use native libraries (thus the user is not the one making the loadLibrary call and therefore cannot change it).

There a few options:

• Let anybody who needs this type of library loading just change it in the Android bindings and let them build and use it themselves (pretty easy but not super maintainable)
• Use a library loader that is able to handle split APKs and all that stuff out of the box (e.g. Facebook's SoLoader)
• Find some condition to determine wether to use System.loadLibrary(...) or SplitInstallHelper.loadLibrary(...) (but I don't know any right now)

I guess this is not really a DeepSpeech specific problem, but I thought I leave it here if someone tries the same as me.

Have I written custom code: Android Project with Dynamic Feature Module that uses DeepSpeech
OS Platform and Distribution: Linux Ubuntu 20.04.

[lissyx]

Let anybody who needs this type of library loading just change it in the Android bindings and let them build and use it themselves (pretty easy but not super maintainable)

Maybe much more maintainable on our side, in fact, but it kills the ease-of-use

Use a library loader that is able to handle split APKs and all that stuff out of the box (e.g. Facebook's SoLoader)

Not a big fan of adding third-party deps, it's going to draw issues of its own

Find some condition to determine wether to use System.loadLibrary(...) or SplitInstallHelper.loadLibrary(...) (but I don't know any right now)

You link seems to give some hints, but I don't know about this splitted feature, so I'm unsure about the behavior of the code

public void onStateUpdate(SplitInstallSessionState state) {
    if (state.sessionId() == mySessionId) {
        switch (state.status()) {
            case SplitInstallSessionStatus.INSTALLED:
                // Updates the app’s context as soon as a module is installed.
                Context newContext = context.createPackageContext(context.getPackageName(), 0);
                // To load C/C++ libraries from an installed module, use the following API
                // instead of System.load().
                SplitInstallHelper.loadLibrary(newContext, “my-cpp-lib”);
                ...
        }
    }
}

Woudnd't this be ran to explicitely detect we have a split install session ?

[lissyx]

Doh, SplitInstallSessionStatus depends on Google Play? That would mean we make libdeepspeech tied to the Play Store API ? That does not sounds super-good.

@erksch Given that we want the bindings to stay as dumb and as clean as possible, wouldn't it make sense to have a wrapper on top of it for the usecase of Split APKs ? We would, I guess, need to adjust the call we have on our side, but if you write wrapper that properly loads using SplitInstallHelper.loadLibrary() then the symbols should be found by the linker when required.

[lissyx]

FTR, we already have higher-level DeepSpeech usage in https://github.com/mozilla/androidspeech/, so it would not be so uncommon.

[Erik Ziegler]

need to adjust the call we have on our side

An idea would be that you wrap the System.loadLibrary calls with a try catch and allow them to fail with a loud warning but thus allowing for a user of dynamic feature modules to just call

SplitInstallHelper.loadLibrary(context, "deepspeech")
SplitInstallHelper.loadLibrary(context, "deepspeech-jni")

right before calling any DeepSpeech functionality.

Would look like this:

public class DeepSpeechModel {

  static {
    try {
        System.loadLibrary("deepspeech-jni");
        System.loadLibrary("deepspeech");
     } catch (UnsatisfiedLinkError e) {
        println("Libraries not found")
        // or for more android style
        // Log.d("DeepSpeech", "Libraries not found");
     } 
   } 

and then in usage

SplitInstallHelper.loadLibrary(context, "deepspeech")
SplitInstallHelper.loadLibrary(context, "deepspeech-jni")
DeepSpeechModel model = new DeepSpeechModel(...)

[lissyx]

If you can confirm it works this could be a start, although I'd prefer we have a way for the user to say its okay to fail, so that people disable it on purpose.

[Erik Ziegler]

Hmm yeah since the libraries are loaded in the static block we can't add a parameter like allowLibraryLoadFail without moving the loading to the constructor. But loading libraries in the constructor would not feel right.

[lissyx]

Maybe an env var ?

[lissyx]

need to adjust the call we have on our side

An idea would be that you wrap the System.loadLibrary calls with a try catch and allow them to fail with a loud warning but thus allowing for a user of dynamic feature modules to just call

SplitInstallHelper.loadLibrary(context, "deepspeech")
SplitInstallHelper.loadLibrary(context, "deepspeech-jni")

right before calling any DeepSpeech functionality.

Would look like this:

public class DeepSpeechModel {

  static {
    try {
        System.loadLibrary("deepspeech-jni");
        System.loadLibrary("deepspeech");
     } catch (UnsatisfiedLinkError e) {
        println("Libraries not found")
        // or for more android style
        // Log.d("DeepSpeech", "Libraries not found");
     } 
   } 

and then in usage

SplitInstallHelper.loadLibrary(context, "deepspeech")
SplitInstallHelper.loadLibrary(context, "deepspeech-jni")
DeepSpeechModel model = new DeepSpeechModel(...)

Dumb question, but have you tried this code ? It could be that the linker will see the objects are already loaded and this way of loading could be enough?

[Erik Ziegler]

Dumb question, but have you tried this code ? It could be that the linker will see the objects are already loaded and this way of loading could be enough?

You may be just completely right. I just tested it using the public DeepSpeech dependency and it works with just loading the libraries beforehand. I guess I was confused thinking that loadLibrary will always fail and because the guides state "use the API instead of System.load()". Thank you :D

[lissyx]

@erksch If you think it's worth doc or example somewhere, don't hesitate to share a PR, otherwise I guess we could just close this ?

[Erik Ziegler]

I think this issue is doc enough, so we can close :)

[lissyx]

I have no idea how much those splitted APKs usages are common, I'd like to avoid relying on people finding a closed issue on GitHub to know how to avoid this, that's all :)