Use native methods to expand the Java environment

The JNI provides a documented and supported specification that allows programs written in other languages to be called from Java. Calling Java from an application written in another language is often referred to as embedding, and the process requires an understanding of the Invocation API. The JNI also provides a way for applications written in other languages to call Java.

Before we get too deep into our discussion, I’d like to provide some background on how native methods worked prior to the release of the JNI. In short, before JDK 1.1 native methods were difficult to use because:

• The specification did not interface to the Java virtual machine (VM) at runtime for symbol lookup. It was assumed that symbols would be in an include file and compiled in at compile-time. Unfortunately, this is not the way object environments work with dynamic linking. This approach also created an opportunity for out-of-date source files, and prevented dynamic changes to values at runtime.

• The interface for embedding was vague.

• There was no support for pinning (preventing garbage collection and movement of Java objects).

• Java strings were converted to pointers without support for large character sets, making internationalization impossible.

• The nomenclature was terribly confusing.

With the introduction of the JNI in JDK 1.1, native methods have become much easier to use. To help you better understand the JNI, we’ll review portions of the actual JNI spec to explain why you need to use it and what it allows you to do. The italicized items are direct quotes from the JNI specification, and the regular text is the annotation I’ve supplied for clarity. Once you know the “why” and “what” of using the JNI, we’ll focus on the “how.” Note that for brevity’s sake I make no attempt to fully explain the JNI specification. If you feel you need a primer on the JNI, be sure to review the complete specification (and also the tutorial), which is listed in the Resources section of this article.

Why do I need to use the JNI?

• The standard Java class libraries do not support the platform-dependent features needed by the application.

  You might need to set some device-specific properties under program control, or access a device-specific API.

• You already have a library written in another language, and wish to make it accessible to Java code through the JNI.

  You may want to rewrite the necessary application in Java, but this is not possible in all cases. Many application libraries on the ‘Net have taken an enormous amount of time to perfect (we’re talking hundreds of thousands of hours); rewriting them in Java is not likely in the short term.

  Unfortunately, using native methods may not be the best solution. When you use native methods in computation-intensive tasks like tight loops, which occur when you repeatedly call a function, you may find that the process of setting up parameters, calling the method, and getting the return values for each iteration of the loop takes longer than the time it takes to execute the function.

• You want to implement a small portion of time-critical code in a lower-level language such as Assembly.

  You have written a Monte Carlo application that simulates your portfolio, but it runs too slowly. You’ve tried the JIT (just-in-time) compiler, but it just doesn’t do it for you.

What does JNI allow me to do?

• Create, inspect, and update Java objects (including arrays and strings).

  If you are using the Invocation API for embedding you can take a two-dimensional array of strings and pass it to the Java program. If your C program is called from Java, you can create and return an array of floats or an array of strings that will be understood by the Java calling method.

• Call Java methods.

  The ability to call a Java method from C/C++ or Assembler is essential to the Invocation API. These JNI functions allow us to call static and non-static Java methods with arguments.

• Catch and throw exceptions.

  Good programmers raise and catch exceptions. Using C/C++ or Assembler native code to raise exceptions makes for more understandable programs.

• Load classes and obtain class information.

  A foreign-language program can request information about the class being loaded.

• Perform runtime type checking.

  The parameters are checked at runtime, not statically as they were in the previous implementation.

Putting the JNI to use once you know when and why to use it is no small task. Proper use requires that you understand numerous technical details. My advice is to pour over as many examples as possible and read the specification in its entirety. (In fact, it may be helpful to view the pdf version of the spec in another window while you work through this article.) When you feel comfortable with the JNI fundamentals, the rest of this article should fall into place.

Calling Java code from C programs

Note: Depending on which language you are using to call Java, the terminology you use may differ from what I use in this section. Please substitute the appropriate differences (for example, function vs. method vs. subroutine) for the language you’re using. Because I am unwilling to rewrite useful pieces of Java code in C or C++ for my C programs, I often call Java directly from my C programs. Building a C/C++ or Assembler application that calls a Java method is much less involved than writing a C program to be called from a Java application. As you’ll see later in the article, the latter technique requires us to make any modifications to the Java application. In this case, however, we drive the Java application from C, so to speak.

For the C program to communicate with the Java program, we need to use several JNI method calls (more detail on these in a moment). Let’s take a look at the steps required to interface to an existing Java application:

1. Create the Java VM.

   To create the JVM you call JNI_CreateJavaVM(), which returns a pointer to the JNI interface pointer. The calling thread is considered the parent thread. Once you have created a VM, you can call Java methods using the standard calling procedure of getting an object, finding its class, getting the method id, and then calling the method id with the appropriate method.

2. Start the Java application.

3. Pass parameters to the Java application and retrieve parameters from the Java application.

4. Attach to the VM.

   This step is optional and used only if you wish to attach to the VM from another thread. In this case, you will have multiple threads running in your C application, each of which needs to use the Java application (for example, a Java application that provides some service used by many competing processes).

5. Unload the VM.

   The calling thread must unload the VM.

The JNI specification details a concept called embedding. The idea with embedding is to create a Java VM that exists in your address space bound to a single thread of the target OS. (On some systems, specifically those that support thousands of threads, embedding may not be the optimum solution.) “But what about multithreading?,” you ask. Embedding does not do away with multithreading; what it does do, however, is allow the Java threads from the VM to be visible to only the calling thread — unless additional methods are invoked. (This is an advanced concept and will not be discussed further in this article. For additional information, see The Invocation API in Chapter 5 of the JNI specification 1.1.)

We’re going to be working through a fully debugged example, which I encourage you to modify for your own purposes. This example doesn’t do very much other than demonstrate how to:

1. Start the VM.

2. Reference methods in the application associated with the VM.

3. Pass objects back and forth between the two environments. In some cases, the objects may not really be objects — in C, for instance, you might pass in a structure.

The example consists of two parts: InstantiatedFromC.java, a Java class that implements some operation we wish to use from C; and tcl2JavaVM.c, the C program in which we wish to use the Java class. The reason for the name tcl2JavaVM.c is for a future plan to allow TCL/TK to use existing Java classes.

We’ll start by specifying an include file that defines the JNI typedefs and defines for the language being used. (You can find this include file in the $JAVA_HOME/lib/include directory, which is included in the JDK developer distribution.) This include file has enough intelligence to determine if you are using C or C++ and generates the appropriate code.

 
#include <jni.h>

The following variables provide us with a handle to the VM, the default starting arguments, and a debugging aid. I’ll leave it to you to determine how you would specify options — other than the default options — to the VM using the vm_args variable.

JavaVM           *jvm;             /* Pointer to a Java VM */
JNIEnv           *env;             /* Pointer to native method interface */
JDK1_1InitArgs   vm_args;          /* JDK 1.1 VM initialization requirements */
int              verbose = 1;      /* Debugging flag */ 

Not much to explain here; the variables do pretty much what the comments indicate:

• *jvm points to the Java VM that we’ll create.

• *env points to the native method interface of the VM.

• vm_args contains args and other options, such as the CLASSPATH.

• verbose = 1 is a debugging flag that allows debugging information to be turned on and off.

Now let’s take a look at the main entry point of the C program. When the computer begins executing tcl2JavaVM, a Java VM will be started with default arguments.

main(int argc, char **argv ) {
jclass          cls;
jmethodID       mid;
jthrowable      jthr;
/* Set up the environment */
JNI_GetDefaultJavaVMInitArgs ( &vm_args );      // Get the default arguments
                                                    // Look at pages 75-76 of the JNI spec 
vm_args.classpath = "H:/opt/jdk/lib/classes.zip;.";    // Set the classpath 
                                                       // equivalent to typing -classpath 
                                                       // on command line or setting the
                                                       // environment variable 
JNI_CreateJavaVM(&jvm, &env, &vm_args );    // Start the VM

We also allocate some local variables to hold references to Java objects: jclass is a C reference to a Java class, jmethodID is a C reference to a Java method, and jthrowable is a C reference to Java.lang.Throwable. Page 19 of the JNI specification contains an excellent diagram that shows how to reference Java objects from C at the type level.

Next, we need to find the class we want to load.

/* Find the class we want to load */
cls = (*env)->FindClass( env, "InstantiatedFromC" );

The FindClass method causes the Java VM to search for the class specified in the second argument (in this case, InstantiatedFromC). For clarity, I have left out the check for the error condition; be sure you perform the check for cls being NULL — in C, of course, as shown below:

if ( cls != (jclass)0 ) {

Assuming we have found the class, our next step is to get a pointer to the method we are interested in calling.

/* Find the method we want to use */
mid = (*env)->GetMethodID( env, cls, "test", "(I)I" );

It seems reasonable that you have to specify the class, the method name, and its signature. This technique simply replaces the code you would normally use for calling a method with a mechanism for doing it from C. In Java, you simply create a reference to the class, unless the method in question is static, in which case you can call it without creating an instance.

Now we are ready to call the Java method from C. The relevant portion of code in the following snippet is indicated with bold type.

/* Call the method we want to use */
printf("First call to Java returns:%dn", (*env)->CallStaticIntMethod(env, cls, mid, 1) );
/* Check for an exception */ 
if ( hasExceptionOccurred ( env ) != (char *)0 ) {
  printf("Exception has occurred.n");
}

The Java method we are trying to call is shown below. This method returns the input argument +1, as long as the result is not 2. If it is 2, an exception is raised.

public class InstantiatedFromC {
        public int test(int number) throws Exception {
                System.out.println("Number from C: " + number);
                if ( number == 2 )
                        throw new Exception("Exception raised in Java seen in C");
                return ( number + 1 );
        }
}

The code segment shown next is identical to the previous one, except that the argument we provide to the Java method is 2 instead of 1. I wanted to demonstrate how to get exception messages from Java in a C program. In this case, when the Java method returns, we call the C method that attempts to determine if there is an exception. We find that there is indeed an exception. A number of error conditions can occur while we are trying to get the exception, and it is important not to confuse these with the error condition we are trying to examine.

/* Call the method we want to use and raise an exception */
printf("Second call to Java returns:%dn", (*env)->CallStaticIntMethod(env, cls, mid, 2) );
/* Check for an exception */ 
if ( hasExceptionOccurred ( env ) != (char *)0 ) {
  printf("Exception has occurred.n");
}

The C function hasExceptionOccurred( env ) returns 0 if no exception was raised. If there is an error while attempting to process the exception response, the function hasExceptionOccurred will stop execution and will terminate itself with an exit call.

Note: Do not confuse this process with processing the exception from an application’s point of view. Before the application can process an exception response, it must have the exception type, string, and, possibly, the stack at the time of the exception. If the exception is valid and has been converted to C format, the application can process it.

The fatal errors hasExceptionOccurred will exit for are:

• Unable to find the object class for Java.lang.Throwable — We must have this class to take the exception apart.

• Unable to find the method id for getMessage in Java.lang.Throwable — We found the class, but it does not have the method we are looking for.

• Unable to call the getMessage method in Java.lang.Throwable — We have the instance, but there is no valid message.

• Unable to convert UTF (the string format used by the Java VM when mapping to C strings) characters from the String object returned from getMessage — We are unable to complete the conversion due to mapping difficulties.

Processing exception information raised in a Java application called from C

If an exception has occurred, it is important to find out what type of exception it is and the error string it produces, if any. Many badly written programs do not go the extra distance to get exception information. The

tcl2JavaVM.c

program includes a simple function for dealing with exceptions more completely. The function defines some automatics, or variables that are allocated on the stack at runtime and disappear when the function returns. Automatics are used to store information used in the program, as well as to allocate space for data to be fetched from the Java VM. The following automatics, some of which you are now familiar with, are defined by the function:

• jthr is a reference to an instance of Java.lang.Throwable.

• jThrowableClass is the class definition for Java.lang.Throwable.

• mid is a reference to a method.

• errorString points to a Java UTF string.

• jbyte maps C constant byte strings to Java immutable byte strings.

We have demonstrated how to launch a Java application from a C program, and how to pass arguments to the Java application, how to get a return argument from the Java method, and how to deal with exceptions. By mastering these techniques you can reuse Java applications from C programs.

Calling C code from Java for communicating to serial ports

Note: The examples used in the remainder of this article are for serial device RS232 drivers for Solaris and Windows 95/NT. Because of potential security risks, Java does not support dynamically installable device drivers. One of the reasons my Windows 95 is unstable is due to device drivers that are routinely updated on my machine. I have no choice regarding the updates because many programs don’t tell you they are changing your device drivers. Few languages have included device support directly in the language, as it was considered part of the operating system’s responsibility. (The only language I am aware of that provided direct support for devices was Ada.)

As I’m sure many of you are already aware, connecting devices to the Java environment can be quite a challenge. At present, the only way to interface to some devices, such as your favorite MIDI board or that cool video feed from your camera, is to use the support provided with the vendor’s libraries/DLLs. (DLL stands for Dynamic Link Library and is the method for loading additional code at runtime in the Win32 world.) But computer peripherals are not the only devices to consider. What about cash registers, scanners, or the thousands of devices in a factory? Java can only interface to a device if it is IP addressable or appears as a file in your file system.

Note: The Resources section includes a listing for Central Data’s portio, a driver that allows you to use Ethernet to talk to serial ports. The portio package is not you standard IP addressable serial port; rather it appears as a file in your file system. Although many industry analysts and top players believe that eventually all devices will become network addressable, this approach is currently cost prohibitive.

The most commonly used device on the PC last year was the serial port. I was surprised it wasn’t the network, but you don’t plug in/out your network as often as you plug in/out devices from your serial port. Because one of the most precious pieces of real estate on a PC is a serial port, it gets a lot of use. For example, a typical day in the life of my serial port goes like this: I plug in my modem, I unplug my modem; I plug in my Smart Card Reader, I unplug my Smart Card Reader; I plug in my ISDN console monitor, I unplug my ISDN console monitor; I plug in my HUB console monitor, I unplug my HUB console monitor; and so on. That’s a lot of devices, and I cannot use any of them from a Java application. It is possible to design a Java application that allows access to serial ports because many target systems have this support built into the OS. Eventually Java will also provide this type of support. However, if you can’t wait, you can use native methods to interface directly to the serial support routines in the underlying OS.

As you well know, you cannot open files from an applet, but you can open files from a Java application. To access a serial port, the obvious answer is to open the serial port from an application. Unfortunately, the obvious is not always the easiest. You’d think you could simply open the port and proceed with business (opening /dev/ttya for Unix or COMx: for Windows 95/NT), but it’s not that easy.

Unix users get the big break (no surprise here). When I tested this approach on Unix I was able to open the serial port as a file, just as the original designers of Unix intended. On Windows 95, however, it was a whole different ballgame. It appears that the I/O subsystem requires a different interface to open a serial port. But that’s not all. Even if you could access the correct interface, you would come face to face with an even larger problem: Serial devices often require you to adjust parity and baud rate settings. In case you are unfamiliar with communications terminology, baud rate is the rate at which transitions are transmitted. For example, a 28.8K modem is a modem that can send 28,000 bits per second (BPS). Parity has to do with counting of the bits about to be sent and sending a bit that makes the total number of ones or zeros even or odd, hence even and odd parity. You’ll also encounter stop bits, which provide the ability to send some preamble bits to allow receivers to synchronize. Users often change the settings of the serial port when they are moving devices around. What this means for you is that once you open the port you may need to set or change the the baud rate and/or the parity. Quality serial drivers will allow you to access every parameter.

So now assume that we could access the correct interface and open the serial port. The question now is: How do you set baud rate and parity? You can use native methods until the Java VM supports device drivers.

The following steps can be used to build an application in Java that wishes to use some of the abilities of the underlying operating system that are not directly available to the Java VM. We are only going to cover the points that are germane to the serial port interface we are going to build.

1. Design the application and clearly define the interface boundary. For example, if you are going to design an interface to a serial device, you need to decide what you can and can’t do from the Java environment. Don’t just assume that whenever you need to talk to a device, you need native methods. If you just want to read and write bytes to the serial port without changing the baud rate you can do this without native methods on some of the Java VMs. However, if you need to set baud rate and parity, then you would have to use native methods.

2. Write the Java application and declare the native methods. The example we’ll work through in this section goes through these methods in greater detail.

3. Compile the code to produce a class file.

4. Using the class file produced in the previous step, create an h file by executing javah -jni "classname", open the h file in an editor, and copy and paste the signature definitions from the .h file to your C file.

5. Using the signatures generated in step 4, write the native code that will implement the desired functionality. To access the method arguments supplied on the Java method call, you need to use the JNI support functions for accessing the arguments.

6. Create a shared library that contains your program. You can verify that the names you are using are going to be found at runtime by running the following utilities:

   In Unix, run:

   • dump — This utility dumps the symbol information in Unix libraries.

   • nm — An earlier version of dump.

   • LD — In combination with debug, LD provides a description of actions being taken with libraries.

   • truss — Dynamic runtime without need to start a debug version.

   In Windows 95/NT, run:

   • dumpbin — This utility displays everything you ever wanted to see about a DLL.
7. Test and debug your program. If you get an unsatisfied linker exception, your paths or your method signatures are incorrect.

Whenever you implement a native method interface, it is important that you try to reuse existing classes. Serial devices present us with a pretty clear-cut choice for determining what existing classes might be of help. Because we want to write and read to streams, we should model the output to look like a stream. Therefore, it would make sense for us to use the Java classes that implement stream interfaces. Specifically, we’re going to extend Java.ion.InputStream and Java.io.OutputStream.

Luckily for us, these classes were originally designed in quite a clever fashion; you need only replace a few methods to reuse all of the existing code. So with very little work on our part, we have SerialInputStream.java and SerialOutputStream.java. Both of these new classes require that the constructor have a PlainSerial object. The PlainSerial class provides the interface, which we’ll call the Serial Device interface. The Serial Device interface has been abstracted for all serial devices supporting the basic read and write of single bytes, with no time or pacing requirements. Serial devices can have very demanding requirements, which are found most often in real-time environments. For example, some serial lines my require paced output or rapid response within given and predictable time frames. (The Resources section of this article provides several links to articles that discuss real-time issues with Java.)

As you might expect, we need to implement open, close, read, and write methods for the PlainSerial class. The SerialInputStream and SerialOutputStream classes use the native methods shown next by calling the appropriate method.

public class PlainSerial {
        public  int     openResult = -1;
        private final native int OpenDevice( String device, int baud);
        private native void writeAByte( int theFd, int theByte);
        private final native int readAByte( int theFd );
        private final native void close( int theFd );
        public void write( int b ) {
                writeAByte( openResult, b);
        }
        public void close() {
                close(openResult);
        }
        public int read( ) {
                return ( readAByte( openResult ) );
        }
        /**
         *      Open the device by name at the requested baud rate
         */
        public PlainSerial(String device, int baud) throws IOException {
                System.out.println("About to call open");
            openResult = OpenDevice(device, baud);
                        System.out.println("Open result:" + openResult );
                if ( openResult < 0 ) {
                        switch (openResult) {
                                default:
                                        throw new IOException("Error Opening Device " + openResult);
                        }
                }
        }
}

Note that the JNI requires the signatures of the native methods to match the name of the calling class and package. For this reason, you see PlainSerial twice in the signature. The C code does the actual interfacing to the underlying operating system’s serial file. The following code segment demonstrates the method signatures required to interface PlainSerial to the underlying operating system.

/**
 * The descriptor for the device is returned, if the
 * device can be opened. If it cannot then a -1 error
 * is returned.
 * Class:     Java_commerce_PlainSerial_PlainSerial
 * Method:    OpenDevice
 * Signature: (Ljava/lang/String;I)I
 */
JNIEXPORT jint JNICALL Java_Java_commerce_PlainSerial_PlainSerial_OpenDevice
  (JNIEnv *env , jobject obj, jstring tty, jint lBaud) {
/*
 * Write a single byte
 * Class:     Java_commerce_PlainSerial_PlainSerial
 * Method:    writeAByte
 * Signature: (II)V
 */
JNIEXPORT void JNICALL Java_Java_commerce_PlainSerial_PlainSerial_writeAByte
  (JNIEnv *env, jobject obj, jint theFd, jint theByte) {
/*
 * Read a single byte
 * Class:     Java_commerce_PlainSerial_PlainSerial
 * Method:    readAByte
 * Signature: (I)I
 */
JNIEXPORT jint JNICALL Java_Java_commerce_PlainSerial_PlainSerial_readAByte
  (JNIEnv *env, jobject obj, jint theFd) {

Complete the interface to the operating system using the following guidelines:

• On the Unix target, use the open and ioctl calls.

• On the Windows 95/NT target, use the CreateFile methods with SetCommState described in the Windows 95/NT answerbooks for the IDE you are using.

Beyond time-out

Because serial devices often do not respond (either they’re not connected, not turned on, or something else is plugged in into the port), time-out support alone is not enough. To address this situation head on, I have also incorporated a bounded buffer, which provides the ability to read with a maximum limit on the wait time. Although I could have implemented this buffer by using the underlying operating system, I chose not to because that functionality can be performed in Java.

Much of the design and implementation for SynchronizedBuffer comes from Concurrent Programming in Java — Design Principles and Patterns by Doug Lea. Here’s how it works: A thread (producer) continuously reads the input device and when characters arrive they are given to the SynchonizedBuffer object using synchronized buffers to implement controlled access. We can’t have the producer (the thread getting the bytes from the serial port) and the consumer (the application requesting the data) updating the counters at the same time. We also use wait and notify so that the consumer is informed whenever there is data from the producer. Be sure to thoroughly examine SynchronizedBuffer.java, SerialInputStreamThreaded.java, and SerialThreadReader.java for all the details. You can view the entire serial package here.

Conclusion

If you’ve made it to this point, congratulations! You’ve digested quite a lot of information. You should now be comfortable (well, maybe with some more practice) with calling Java from C and calling C from Java. Remember you need not wait (or be enslaved) by the Java VM if it doesn’t allow you to get the job done. However, once you take the plunge with native methods, you should be aware of several disadvantages:

• If you use native methods, you’re not 100% Pure.

• You must maintain multiple versions of your apps for the various target systems that your users will be using. This means serial drivers for Windows 95/NT, Unix, and Macs … and, oh yes, the NC. Your code will not be portable.

• Native methods require more programming skill and time to get them right.

• A malicious attack against your program could introduce a security risk by replacing the native method code.

• It is easier to crash a system with native methods due to a bug.

This article lays the foundation for my upcoming smart card series. In order to use smart cards, you need to talk to them, and one of the cheaper ways of talking to them is with a serial card reader; thus, the need for serial device support. This series of articles will cover the techniques for using a smart card with your applications. We’ll work closely with the Java Electronic Commerce Framework (JECF), which I will use as the foundation layer for the examples we’ll build.

Rinaldo Di Giorgio is a staff engineer for Sun Microsystems in New York City. He currently is working on the use of smart cards in solving business problems.