JNI in Android (and a foreword of why JNI Sucks)

wall-smaller

Quick Preface: You do NOT need root to create and use a shared native library (.so) in Android from your Java application.

I had never tried Java Native Interface until today. And I must say, it completely sucks. Not only is the header generation/implementation nontrivial and tedious, JNI is inherently broken in that it is not architecture/platform nonspecific. So I proudly present to you my list of why JNI sucks.

  • You can't access arbitrary functions in arbitrary DLLs. JNI requires that you write a glue C/C++ layer to do whatever you want to do natively.
  • On Android, your glue layer must contain a JNI_OnLoad function that explicitly register every method that needs to be made available to the JVM. The JVM can't simply check the export tables and intelligently import it.
  • Since developers must write a glue layer, you must also compile and package that glue layer per platform you are targeting. This design is absurd because platforms can have different CPU architectures and different Operating Systems, and still support similar calls in the similar libraries. For example:
    • OpenGL ES (libGLES_CM). On Windows Mobile it is called libGLES_CM.dll. On Linux/Android it is called libGLES_CM.so. These libraries have the same "undecorated" library name, and also contain the same functions with the same signatures. Unfortunately due to how how terrible an implementation JNI is, the same Java application would not work on every platform.
    • Standard System Libraries on Linux or Windows (libc, libdl, libm, ole32, kernel32, etc) - Once again, need to compile libraries per architecture (ARM, x86, x64, MIPS, etc) to use the common/base functions that would be available on the platform (GetWindowEx, statfs, etc).
  • Write once, run anywhere. (Given that you are willing to do multiple cross compiles and embed multiple native binaries into a single package and unpack the appropriate one at runtime.)

For amusement's sake, let's look at what a C# PInvoke looks like versus JNI on Android. Suppose we want to clear the screen in OpenGL:

C#:

Just declare the method signature and the DLL. Simple! Call that like any other method in C#. (Note that the managed executable generated by this code works on both Android or Windows Mobile. Yes, I tested it.)

[DllImportAttribute("libGLES_CM")]
public static extern void glClearColor(float red, float green, float blue, float alpha);

JNI? Let's start out by declaring it.

public static native void glClearColor(float red, float green, float blue, float alpha);

Not done yet! Now the fun begins. Drop to a command prompt and go to the bin directory of your project: javah com.koushikdutta.JNITest.JNITestAcitivity. Now we have a generated header file, com_koushikdutta_jnitest_JNITestActivity.h, that looks like this:

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_koushikdutta_jnitest_JNITestActivity */

#ifndef _Included_com_koushikdutta_jnitest_JNITestActivity
#define _Included_com_koushikdutta_jnitest_JNITestActivity
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_koushikdutta_jnitest_JNITestActivity
 * Method:    glClearColor
 * Signature: (FFFF)V
 */
JNIEXPORT void JNICALL Java_com_koushikdutta_jnitest_JNITestActivity_glClearColor
  (JNIEnv *, jclass, jfloat, jfloat, jfloat, jfloat);

#ifdef __cplusplus
}
#endif
#endif

Ok, now let's implement the C code:

#include <GLES/gl.h>

JNIEXPORT void JNICALL Java_com_koushikdutta_jnitest_JNITestActivity_glClearColor
  (JNIEnv * env, jclass clazz, jfloat r, jfloat g, jfloat b, jfloat a)
{
    glClearColor(r, g, b, a);
}

You probably thought you were done, didn't you? Sun thinks otherwise. You need to register the function in C code as well. Don't screw up when copy pasting the cryptic method signature strings!

static JNINativeMethod sMethods[] = {
     /* name, signature, funcPtr */

    {"glClearColor", "(FFFF)V", (void*)Java_com_koushikdutta_jnitest_JNITestActivity_glClearColor},
};

extern "C" jint JNI_OnLoad(JavaVM* vm, void* reserved)
{
    JNIEnv* env = NULL;
    jint result = -1;

    if (vm->GetEnv((void**) &env, JNI_VERSION_1_4) != JNI_OK) {
        return result;
    }

    jniRegisterNativeMethods(env, "com/koushikdutta/JNITest/JNITestActivity", sMethods, 1);
    return JNI_VERSION_1_4;
}

To be completely fair, jniRegisterNativeMethods is actually a convenience function that is found in libnativehelper.so on Android; I imported that library for the sake of... convenience. Its implementation is as follows (JNIHelp.c in the source):

int jniRegisterNativeMethods(JNIEnv* env, const char* className,
    const JNINativeMethod* gMethods, int numMethods)
{
    jclass clazz;

    LOGV("Registering %s natives\n", className);
    clazz = (*env)->FindClass(env, className);
    if (clazz == NULL) {
        LOGE("Native registration unable to find class '%s'\n", className);
        return -1;
    }
    if ((*env)->RegisterNatives(env, clazz, gMethods, numMethods) < 0) {
        LOGE("RegisterNatives failed for '%s'\n", className);
        return -1;
    }
    return 0;
}

You need to load the library in Java code in your class's static constructor.

System.load("/data/data/com.koushikdutta.JNITest/libjnitest.so");

On Android, you must use System.load and provide a full path instead of using System.loadLibrary. This is because loadLibrary only checks /system/lib; it does not search LD_LIBRARY_PATH or your current working directory. Unfortunately, applications can't write to the /system/lib directory (without root).

.NET's P/Invoke is so much easier/better that a third party company (God forbid Sun actually do something to better the Java language) implemented something quite similar for Java, named, *drumroll please*, J/Invoke. Too bad Android doesn't have that. That would be nice.

Also, on Android, you can't do a normal ARM/GCC cross compile to make shared libraries (I've discussed this in a previous blog post). The resultant .so files will not work with Android's linker due to different linker scripts. To create shared libraries for Android, I would highly recommend just creating a subproject in the Android Source code tree and building the tree. Alternatively you can use the handy script found on this page.

Also, remember that the shared library will need to be contained in your APK file and extracted to the file system so it can be accessed. For an example of how that can be done, check out the source code that does something very similar in the Android Mono port.

Incidentally, this learning experience came about as a result of figuring out how to run Mono side by side with Java/Dalvik in the same process.

21 comments:

OneShare said...

nice blog & very informative, i'm bookmarking this blog & visiting again for updates.

please check mine too : FREE Comics Download

shamaz said...

JNI is indeed quite old, and has not evolved very much for the sake of compatibility...
But an alternative has been proposed :
https://jna.dev.java.net/

You should take a look... but I don't know if it would work with Dalvik.

shamaz said...

Oh actualy, it seems JNA is not supported yet on Android :
http://markmail.org/message/eq7seicibmfwazuy

James said...

Fuckin awesome. I love reading this shit. Keep it up.

Dominic said...

Really, no one writes JNI code by hand these days. Use SWIG if you have to. Or libffi/JNA if you want something akin to C#'s DllImport. If JNA doesn't work on Android, that's more the fault of Google's custom VM and runtime than of Java as a language.

Andy Bee said...

With this JNI method, Which capability are there that you can do with G1?
I mean that what we can do more than using only java.

Beowolve said...

As far as I know you the .so files are automatically placed in the system folder if you install a apk file.

AØT & KLM said...

good work! but you don't actually need to register functions, if you export them with extern "C", the jvm will find the functions automatically in the .so file.

Andy McFadden said...

Re: System.load() vs. System.loadLibrary

This changed when native development became officially supported several months ago. With the release of the NDK, the shared library search path includes an app-private directory into which shared libs are placed at app install time. There is no need to use an absolute path.

Re: JNA

It's more convenient than JNI for existing libraries, but at a price. From the JNA FAQ:

"The calling overhead for a single native call using JNA interface mapping can be an order of magnitude (~10X) greater time than equivalent custom JNI ..."

The expectation on Android is that, if you're writing native code, you're most likely doing it for performance reasons. This is not always the case, and something like JNA would no doubt be useful for many developers, but making it (or something like it) available is lower priority than many other things.

The JNA sources are available and are relatively VM-agnostic, though it may be necessary to stub out the AWT window management features. There's no reason why this couldn't be done by an external developer and published under the LGPL.

Re: JNI_OnLoad

As a previous poster noted, explicit method registration is not (and has never been) required. You do not need to have a JNI_OnLoad method. It's recommended though, for reasons laid out in the "JNI Tips" document in the Android source tree.

pzul said...

Yeh, JNI is a bit tedious - but aren't Java programmers already used to extreme verbosity?

Last time I checked Android was the only phone that supports callout from Java. The Java on most phones (J2ME, CLDC, BREW) does not provide any callout at all.

For me the introduction of JNI on mobile Java was a cause for celebration because before that it was much worse. The only thing you could do was implement a network service on your phone and have your Java code send messages via localhost.

Anonymous said...

-C++ JAVA binding is horrible.
-Parcel container is a nonsense in an OO medium.
-SurfaceFlinger & UI libraries are written at the mental hospitals across the country.

Anonymous said...

Nice post. Here is the class diagram for surface flinger. A 2D engine that has to draw up to 31 rectangles onto screen. Damn !, how they can publish something like that! Is this the new engineering era, the new java engineers, that write C++ code?

http://docs.google.com/present/edit?id=0ATd_esqYSHtAZGN6ZmdjcnZfM2dxcmpudGdm&hl=en

Jon Shemitz said...

I think the interesting question is not how much easier P/Invoke is than JNI but how the two compare in terms of run-time cost. After all, .NET was largely designed for a corporate development environment where programmer time is much more expensive than CPU time. It makes sense to bake convenience into the CLR. With 1GHz / 512M being a high-end device in Q3 '10, mobile code clearly runs in an environment where CPU time is more precious than programmer time.

I think this is somewhat similar to the difference between the horrors of (the Binder and) AIDL vs the simplicity of (.NET remoting and) transparent proxies. It's worth a few hours of programmer pain to make app launch noticeably faster.

(Please do note that I have not done any of these benchmarks. I'm just getting up to speed on low-level Android stuff, and ran into this blog looking for info about linking libraries against Bionic.)

nick said...

I'm not sure how you can gripe about JNI not being easy to use when switching between .so and .dll and then use a windows native example as an "easier way".

Anonymous said...

Here's another way that it's broken: the implementations that I am familiar with work by using a jump instruction, and running whatever instruction it finds when it gets there. This effectively blocks all threads.

Paul Danger Kile
dangerismymiddlename.com

Anonymous said...

JNI is a real low-level interface with everything that entails, unlike P/Invoke. Even you, with no knowledge of low-level programming, should know that. Apparently, you have no idea what you're talking about.

Windows dumbasses and .NET programming hacks always amuse me. Try to get it in your head that Java *is* multiplatform, C# isn't.

JNI invocation of native code works on EVERY platform Java runs on. C# is Windows-only. There are open source alternatives to .NET runtime/compiler, like Mono, but Mono has nothing to do with Microsoft or NET. NET P/Invoke is a Windows-only extension and is absolutely non-portable in the sense you're implying.

Native code is native code. That's what it means. When you want to support several architectures, you have to compile for each of them.

You're not even comparing apples to oranges. Not knowing much about Java and JNI doesn't excuse this kind of ignorant ranting. When you know shit, try to keep it to yourself.

Let's not forget about Java's JNI alternative, JNA, that you apparently know nothing about. It is intended exactly for lame low-level programmers like yourself.

Which subset of Java specs Google's Dalvik supports isn't SUN's decision, you noob. And last but not least, try to look up the standard javah tool that generates the "JNI glue" for you.

Man, why am I even wasting time to write this...

Koushik Dutta said...

To the poster above me.

Mono lets you P/Invoke native on any platform. That means linux on mac as well.

Let me reiterate, you can invoke native code, on any platform, without having to write native code glue. .NET lets you run TRUE platform agnostic code, because you never need to write native code. It is NOT windows only.

I've written significant amount of JNI. Probably moreso than you. How do you think I implemented the Mono/Java bridge for android?

https://github.com/koush/androidmono/tree/devel/MonoActivity/jni

Your ignorance is showing.

Anonymous said...

Hahaha. Two comments up must have stumbledUpon this or something. Just read this from your Twitter post Koush. Thanks for the laugh.

Sami said...

C# runs on Windows only? Did you know that .net framework exists on linux and mac either? made my day

Dmitry Skiba said...

I wrote a library which makes JNI suck less: https://github.com/DmitrySkiba/itoa-jnipp

True, it requires you to define all classes and method you need, and you define them through a set of voodoo macros, but once you've done, you can write pretty decent C++. Check the examples: http://dmitryskiba.github.com/itoa-jnipp/examples.html

Paul Danger Kile of DangerIsMyMiddleName.com said...

JNI's suckage is epic.

When your program gets to the native block it does a JUMP instruction to the native code's location. From there you can imagine some Assembler, or machine instructions, depending on how you imagine such things, chugging along. So, where am I going with this? JNI blocks _all_ of your JVM's threads. Really.

Way back when, folks were publishing JNI articles and examples, and advising programmers to use it in order to increase throughput on your server, but it actually inhibits throughput on servers, due to the blocking.

It's a whole different game in the non-server world. It _might_ have a purpose there. You could wrap a native library. That takes a lot of work, but that's a use for it.