Attempt to remove non-JNI local reference with GlobalObject

[therealkenc]

When using jni-bind with Android Studio, I am getting a warning in my logs. Looks like:

Minimal repro:

JNIEXPORT void JNICALL
Java_com_example_fssrv_MainActivity_sendGreeting(JNIEnv *, jobject, jstring jmsg,
                                                 jobject jcb) {
    static constexpr jni::Class kGreetingCallback{
            "com/example/fssrv/GreetingCallback",
            jni::Method{"invoke", jni::Return<void>{}, jni::Params<jstring>{}}
    };
    jni::LocalString msg(jmsg);
    // absent this Release() below there are two "Attempt to remove non-JNI local..." warnings
    msg.Release();
    jni::GlobalObject<kGreetingCallback> cbo{jni::PromoteToGlobal{}, jcb};
    //cbo.Release();   // <-- doesn't matter Release() here or not
}

My real code is basically replicating the pattern in this test case. Meaning, whether one was to stick a new() on that GlobalObject construction makes no difference.

Checking the Interwebs I found this comment. "If you're a developer getting this message in your own app, make sure you're not accidentally deleting local references given as parameters to your JNI methods."

The jni-bind README states:

Because jobject does not uniquely identify its underlying storage, it is presumed to always be local

While the object always be a local, like the LocalString in the minimal example above, it isn't necessarily our local on which we can do an env->DeleteLocalRef(). I suspect but can't prove that is what is happening with jobject jcb above. Or, at the very least, if I comment out the one line promoting jcb to global, the warning disappears.

The code appears to function well enough with the warning, and indeed I didn't notice until I started porting my Java language binding from Desktop to Android. It is tempting to just ignore it, but this is a protocol thing so there are going to be hundreds or thousands a second.

[Jamieson]

This feels like a deficiency of JNI Bind. LocalObject always presumes itself to have ownership, and you can express that it loses it by Releaseing it, but really you ought to be able to say it at construction that it should never be deleted.

I've had it in my head to have a NoDeleteLocal (or something better named) that would be used for "performance mode", where locals by default don't delete themselves. That said, as you've pointed out, it's not just a speed thing, sometimes you want the syntactic sugar of JNI Bind, but you also know you don't own the object.

--

I think something like this would be good:

LocalObjectND<kClass> obj { obj };
LocalStringND str { "Foo" };

I think there probably should be ways to convert this into a local (in the same way globals have mechanisms for locals), however, I think as a first pass, I might be able to make something useful quickly.

I'll take a quick detour from solving your Windows problem as this might be easy (and I'd like it pre 1.0). Apologies that has taken so long, but as a non Windows dev it probably takes me 5x that amount of time to do things as you.

--

As to why you''re not seeing failure, I agree, you're probably violating the contract. I've seen endless idiosyncrasies like this, e.g. you can call DeleteLocalRef on objects passed in as many times as you want, but if natively constructed it will crash (which might explain why your code is working no problem).

The only certain way feels like properly obeying the contract.

[therealkenc]

I managed to address the "Attempt to remove" with NewGlobalRef() and AdoptGlobal{}

extern "C" JNIEXPORT void JNICALL Java_com_example_fssrv_MainActivity_sendGreeting(
    JNIEnv*, jobject, jstring jmsg, jobject jcb)
{
    static constexpr jni::Class kGreetingCallback{ "com/example/fssrv/GreetingCallback",
        jni::Method{ "invoke", jni::Return<void>{}, jni::Params<jstring>{} } };
    jni::LocalString msg(jmsg);
    std::string greeting{ msg.Pin().ToString() };
    msg.Release();
    greeting += ", with stuff from sendGreeting() ❤️";

    using CallbackObject = jni::GlobalObject<kGreetingCallback>;
    auto gjcb = jni::JniEnv::GetEnv()->NewGlobalRef(jcb);
    CallbackObject cbo{ jni::AdoptGlobal{}, gjcb };
    cbo("invoke", greeting);

    std::thread([greeting, cbo = std::move(cbo)]() mutable {
        jni::ThreadGuard guard = jvm->BuildThreadGuard();
        std::this_thread::sleep_for(1s);
        cbo("invoke", greeting);
    }).detach();
}

I am about 90% (but not 100%) sure the call to the "invoke" callback inside the thread will crash sometimes on Android if the local reference is removed. Of course it won't crash for me today, so no repro, and I've never seen it crash on Desktop. I am not actually sure what the "dumping thread" in the warning means, but it doesn't sound good.

[therealkenc]

I think there probably should be ways to convert this into a local

I built some wrappers to make the code more expressive.

namespace jni
{
template <typename Callable, typename... Args>
std::thread thread(Callable&& fn, Args&&... args)
{
    return std::thread(
        [fn = std::forward<Callable>(fn), ... args = std::forward<Args>(args)]() mutable {
            jni::ThreadGuard guard = jvm->BuildThreadGuard();
            fn(std::move(args)...);
        });
}

std::string string(jstring jstr)
{
    jni::LocalString lstr(jstr);
    std::string ret{ lstr.Pin().ToString() };
    lstr.Release();
    return ret;
}

template <const auto& class_v, const auto& class_loader_v = jni::kDefaultClassLoader,
    const auto& jvm_v = jni::kDefaultJvm>
constexpr auto adopt_global(jobject obj)
{
    auto gobj = jni::JniEnv::GetEnv()->NewGlobalRef(obj);
    return GlobalObject<class_v, class_loader_v, jvm_v>{ jni::AdoptGlobal{}, gobj };
}
} // namespace jni

Function then becomes:

extern "C" JNIEXPORT void JNICALL Java_com_example_fssrv_MainActivity_sendGreeting(
    JNIEnv*, jobject, jstring jmsg, jobject jcb)
{
    static constexpr jni::Class kGreetingCallback{ "com/example/fssrv/GreetingCallback",
        jni::Method{ "invoke", jni::Return<void>{}, jni::Params<jstring>{} } };

    auto greeting = jni::string(jmsg) + ", with stuff from sendGreeting() ❤️";
    auto cbo = jni::adopt_global<kGreetingCallback>(jcb);
    cbo("invoke", greeting); // needed to work-around #190
    jni::thread([greeting, cbo = std::move(cbo)]() mutable {
        std::this_thread::sleep_for(1s);
        cbo("invoke", greeting);
    }).detach();
}

I am about 90% [...] sure the call to the "invoke" callback inside the thread will crash sometimes on Android

Lowering my probability estimate to fifty-fifty. I think the crashes might have been #190. Now I'm thinking that the cache was being warmed up on Desktop from the main (read: UI) thread, and I misattributed the fail to the warning I saw in the logs on Android.

[Jamieson]

I took a stab at trying to implement a NoDelete type quickly, but frustratingly I ran into issues with the way method lookup works. The alternative would be to create copies of every Local implementation and implement it (which is possible, and perhaps will still happen) but I think sadly it's not the quick fix I was hoping for.

I think, for now, since Release does still work, despite being syntactically inferior, I'm not going to prioritize this. I will capture a Passthrough class I created so I can revisit this later, but I need to think on what's best a bit more.

As for you helper functions, I just realised I haven't put CreateCopy in the documentation, but I think it does what you're doing above.

[Jamieson]

Sorry, I'm unsure if I can mark this complete or not, it sounds like CreateCopy was what you needed, but I'm afraid for now the best I can do is guidance for the "non owned Local". I do think this should be possible, but for now it's just a little difficult to implement without being messy, and the Release or Copy route does work.

[therealkenc]

The problem is not (just) the LocalString. The more serious problem is:

jni::GlobalObject<kGreetingCallback> cbo{jni::PromoteToGlobal{}, jcb};

That's the minimal repro. One line. If I don't handle the LocalString with kid gloves as well, it is two warnings.

Release or no release of the GlobalObject (sic!), this results in a "Attempt to remove non-JNI local reference" warning on Android. I suspect desktop too; I believe the difference is Google turns on some strict JNI logging (or something).

The work around I am using is:

    using CallbackObject = jni::GlobalObject<kGreetingCallback>;
    auto gjcb = jni::JniEnv::GetEnv()->NewGlobalRef(jcb);
    CallbackObject cbo{ jni::AdoptGlobal{}, gjcb };

Please confirm: Is that the correct work-around for now? At minimum, it violates this from the README:

When possible try to avoid using raw jobject.

That jcb is a raw jobject. I'll buy if this is a "not possible" situation I guess.

[Jamieson]

I understand what's happening here I think. While not on the public repo, I do have android_instrumentation_tests internally, and I see this appearing in our logs also (I would export them but I couldn't get the environment to work on GitHub).

The issue is that JNI objects passed into the JNI call have local mechanics, but are not "deletable". Android appears to treat them differently than plain Java (or Java is deciding to surface the failure in a silent way). If you delete a passed in jobject, it gives the failure you are identifying above,

To test this, I simply capture an input argument with LocalObject and I see the failure (i.e. LocalObject<kClass>(obj);). If I do LocalObject<kClass>(CreateCopy{}, obj);, I do not see the failure.

With your Global examplePromoteToGlobal is required to delete the underlying LocalObject it's promoting. If you use CreateCopy, you will be OK. To verify, I called the following JNI code line by line and see the failures (or not):

JNIEXPORT void JNICALL
Java_com_jnibind_test(JNIEnv* env, jclass, jobject object) {
  // jni::LocalObject<kObjectClass>  obj1{object};  // fails, releases unowned `object`
  // jni::LocalObject<kObjectClass>  obj2{};  // passes, releases owned implicit jobject
  // jni::LocalObject<kObjectClass>  obj3{jni::CreateCopy{}, object};  // passes, creates local deletable copy
  // jni::GlobalObject<kObjectClass> obj4{jni::PromoteToGlobal{}, object}; // fails, deletes unowned `object`
  // jni::GlobalObject<kObjectClass> obj5{jni::CreateCopy{}, object};  // passes, creates local deletable copy
}

I clearly need to add documentation for CreateCopy ASAP. Does this guidance leave you in an OK state though?

[therealkenc]

// jni::GlobalObject obj5{jni::CreateCopy{}, object}; // passes, creates local deletable copy

I'm too slow to know what that comment means. I don't want a local anything, nor a copy of anything. It could be that I am reading too much into CreateCopy{} semantics. I am looking for a NewGlobalRef of object.

Is obj5{jni::CreateCopy{}, object} the equivalent of:

    auto gobj = jni::JniEnv::GetEnv()->NewGlobalRef(object);
    jni::GlobalObject<kObjectClass> obj5{ jni::AdoptGlobal{}, gobj };

?

Either way, I did try:

extern "C" JNIEXPORT void JNICALL Java_sparx_jniwrapper_FileServiceLibrary_sendGreeting(
    JNIEnv*, jobject, jstring jmsg, jobject jcb)
{
    static constexpr jni::Class kGreetingCallback{ "sparx/jniwrapper/GreetingCallback",
        jni::Method{ "invoke", jni::Return<void>{}, jni::Params<jstring>{} } };

    auto greeting = jni::string(jmsg) + ", with stuff from sendGreeting() ❤️";
    //auto cbo = jni::adopt_global<kGreetingCallback>(jcb);
    jni::GlobalObject<kGreetingCallback> cbo{jni::CreateCopy{}, jcb};
    cbo("invoke", greeting); // needed to work-around #190
    jni::thread([greeting, cbo = std::move(cbo)]() mutable {
        std::this_thread::sleep_for(1s);
        cbo("invoke", greeting);
    }).detach();
}

If that usage is (scary quote) "correct" on your say-so, that is good enough for me. I can confirm the above does not result in the "Attempt to remove" warning and does work.

Can't say I fully grasp how to use jni::CreateCopy{} in conjunction with jni::LocalString, or put another way, I still don't grok the correct way to get a std::string out of a jstring parameter passed to us. Right now I am doing:

    jni::LocalString lstr(jstr);
    std::string cstr{ lstr.Pin().ToString() };
    lstr.Release();

If/when you are doing the doc updates, some guidance would be helpful.

[Jamieson]

I will update the documentation as soon as possible.

re:
// jni::GlobalObject obj5{jni::CreateCopy{}, object}; // passes, creates local deletable copy

I could have phrased this better. If you give a GlobalObject a jobject with PromoteToGlobal, it will attempt to delete the local it was given (if you're promoting it then you should delete the thing you began with). CreateCopy effectively does not do this, it creates a copy of a LocalObject (which is "deletable", unlike the jobjects passed into JNI).

re: your code block, I don't think you need your jni::string helper method. LocalString and GlobalString have char* construction.

Your CreateCopy usage looks good.

I think it might make CreateCopy the default behaviour since this use case feels like the obvious default. I'm going to take a look and see if it's feasible.

[therealkenc]

re: your code block, I don't think you need your jni::string helper method. LocalString and GlobalString have char* construction.

I don't have a char*, I have a jstring jmsg that isn't mine.

extern "C" JNIEXPORT void JNICALL Java_sparx_jniwrapper_FileServiceLibrary_sendGreeting(
    JNIEnv*, jobject, jstring jmsg, jobject jcb)

I realize, acutely, that it is a stupid quesiton, but what is the idiomatic method to get a std:string from a jstring? The helper is shorthand for:

    jni::LocalString lstr(jstr);
    std::string cstr{ lstr.Pin().ToString() };
    lstr.Release();

I think it might make CreateCopy the default behaviour since this use case feels like the obvious default.

Throwing it out there, fwiw, I am not clear on the use case for PromoteToGlobal{}. If the java object is not ours, we can't "delete the thing we began with" without JNI whining (at least visibly on Android). And if we wanted a global object that is our own, we'd have just constructed a global to begin with.

Making CreateCopy "the default" would be good, as it would steer consumers in the right direction. That said, the name is counterintuitive. It isn't a copy we want. It's a new global reference. If CreateCopy{} means new reference, awesome, but is not immediately obvious.

[Jamieson]

Yah, I agree it's hard to understand, but I unfortunately need it for the case of "no really, I have a local jobject that wasn't passed into JNI that I want to be a global". It's more for when you have to interact with the porcelain JNI.

I agree CreateCopy is a bad name, it perhaps should be called NewReference. I've made a some good progress on making this the default, so far E2E testing is all passing without the error above, I just need to update unit tests.

As for the "idiomatic" way, I have used blocks like this:
printf( "Value:%s\n", LocalString {my_jstr}.Pin().ToString().data());

Is that not working as expected for you?

[therealkenc]

As for the "idiomatic" way, I have used blocks like this:
printf( "Value:%s\n", LocalString {my_jstr}.Pin().ToString().data());

Is that not working as expected for you?

It's not my jstr. From the top:

    jni::LocalString msg(jmsg);
    // absent this Release() below there are two "Attempt to remove non-JNI local..." warnings
    msg.Release();

I don't think you need your jni::string helper method.

I am just going to go ahead and assume I need the helper; there is only so many ways I can ask the question. Do appreciate you help; looking forward to the 1.0 release.

[Jamieson]

Alright, I've finally had a chance to make some progress on this. I cut a new 0.9.8 prerelease which makes NewRef (formerly CreateCopy) the default and I added some documentation introducing NewRef.

Sorry to take so long.