Monitor your Web server in real time, part 3

This is the last and final installment in the Percollator series. The Percollator applet has been developed over several columns in JavaWorld. The data for Percollator is generated by Percollator.se described in the companion article. In this article we will discuss the following additions to Percollator:

• Addition of a Gantt-like chart that displays the status of the rules as reported by SE tool — a tool for monitoring Solaris Operating Systems with rule based metrics, recently enhanced to support Internet metrics. Here is a snapshot that shows what it looks like. (We provide this and other screen shots in GIF format for readers who do not have Java VM ability).
• • Performance measurement of the above using the new Guava compiler
  • Improved appearance of the User Interface and new summary with nine graphs. Here is a snapshot.
• Integrating persistence into the Percollator applet.
• • Example background process that takes rule data and saves it in a persistent database.
  • Using a calendar-like interface to navigate. (Here is a snapshot.)
• A collection of status indicators for your desktop. (Here is a snapshot.)
• Update on VRML  representation of the data.

Use the Percollator applet to monitor a JavaWorld server. (Here is a snapshot.) The development kit for Percollator provides the main classes that can be interfaced to many component libraries.

Gantt-like chart displays the status of the rules as reported by SE tool

The SE perforamnce Toolkit provides the ability to set rules that can be used to classify the behavior of a system into zones, Red — busy, Green OK. For more information on how to setup and use rules look at the following two articles:

• Introduction to Percollator
• Advanced Rule Setting

The RuleStates applet produces a chart that allows a systems administrator to quickly see the status of all the rules specified in relation to one another. The horizontal axis represents increasing values of time, and the vertical axis shows rule and its respective status.

Performance measurement

The Guava runtime demonstrated some significant speed improvements over the standard JDK runtime for

RuleStates.java

. The results of running the test on a data set containing 911 samples are shown in the table below.

The Guava compiler provides a dramatic increase in performance and there are many applets that will benefit from this technology. Guava has implemented an efficient threading model with the its Java VM, but Guava currently does not support native methods. To improve the performance of your Java programs/applets, first calibrate your client or server by running something like CaffeineMark. Then try to understand where the time is being spent in your applet/application by using tools like ProfileViewer. Then try the various JITs on the market. Using the Guava JIT also improves your compilation speed. I am not endorsing any of the above tools. I have been able to use JITs to improve performance often.

Integrating persistence into the Percollator applet

The following discussion expands on my recent JavaWorld article, “RMI and object serialization,” which discussed persistence and introduced a simple object database. The serialization examples discussed there provided directions for installing the RMI libraries. These directions need to be followed again. Don’t despair: RMI will be included with JDK 1.1.

Percollator can be directed to read data sets from the server with the Load menu. When it is directed to do so, the data set is read, and the data is stored in an object that is created from the class DataSet. This class contains the binary representation of the input file, including some statistics on the data such as totals, minimums, and averages. Recall that the server being monitored is producing a log file with samples at the specified sampling rate. These samples are then read by the Percollator applet. Reading this URL each time and performing all the conversions is not necessary if we use the serialization abilities of Java which we discussed previously. Serialization allows us to save Java objects to streams and to reuse them at a later point in time. Percollator was modified so that it can read DataSet objects directly — without having to read the ASCII log file and recreate the DataSet object. An additional program called Save was written to create a DataSet object based on an output log file created by the log program. Save is required on the server to take the ASCII log files and save them as Java objects. You can modify this to actually save Java objects in a database or provide your own interface for retrieval. The calendar panel was modified so that when the panel is first created, the class queries the server to see if an entry exists for the requested data.

The persistent database class library I am using supports the following methods:

• get — get a serialized object off the server’s disk into your applet/application.
• put — save an object in your applet/application on the server’s disk.
• delete — delete the object on the server.
• exists — determine the existence of the object.

The Server program is constantly monitoring some user-specified port for service requests. The Server program manages the database by using a hashed index file that contains pointers to files containing the serialized object. When the Server program is started, you can specify a directory in which to save the index and the serialized objects, as well as a port number to monitor. The server should be running the following application:

java PersistentDB -d /PercollatorObjectDatabase -p 8000 -f index

The above line starts the Object database listening on port 8000 and creates an index file (called index) that can be considered the database name, in directory /PercollatorObjectDatabase. This file also contains the mapping of individual names to serialized files which are Java Objects that have been written to an output stream and saved in a disk file.

Background process saves data into the persistent database

The following Java application must be run on the server to create an object and place this object in the persistent database. This must be done for each log file you which to have in the database.

java SaveDataSetObject “name of Percollator Log file”

This application should be run only by the systems administrator or some Unix cron job (a Unix program that runs regularly at administrator-specified times), and should be run after the data collection for the interval you are interested in has terminated. In this case, the application is being run after midnight on a daily basis to save the previous day’s output in the database.

These methods expect to communicate with a server on a known port. The following code example shows how one would save a DataSet object on the server in the database.

import java.applet.Applet;
import java.awt.*;
import java.net.*;
import java.io.*;
import controls.*;
/**
 <b><i>*</i></b> <tt>Allow the user to specify a Percollator input file  
 *      that will be encapsulated with an object of type DataSet that will
 *      contain the normalized data set.
 *      java SaveDataObject "name of file"
 *
</tt>*/
class SaveDataSetObject {
        ////////////////////////////////////////////////////////////
        //      Open the requested data set and save it as an object
        ////////////////////////////////////////////////////////////
        public static void main(String args[]) {
                long    snapTime = 0;
                if ( args.length != 1 ) {
                        System.out.println("Usage: name of file to save in Object
Store");
                        System.exit(0);
                }
                DataSet ds = new DataSet();
                File    fn = new File(args[0]);
                snapTime = System.currentTimeMillis();
                if ( 
<b>ds.readDataSet</b>
(null,args[0],true,null) == -1 ) {
                        System.out.println("Error Reading file");
                }
                System.out.println("Time to read file is:" +
(System.currentTimeMillis() - snapTime));
                ///////////////////////////////////////////////
                //     
<b><i>Compressing the records
</i></b>
                /////////////////////////////////////////////// 
                System.out.println("Compressing:" + ds.recordNumber);
                ds.compress();  
                ///////////////////////////////////////////////
                //      
<b><i>Save the read in data set as an object
</i></b>
                /////////////////////////////////////////////// 
                System.out.println("Saving Object " + ds.recordNumber);
                PersistentMixin permixin = null;
                String          ServerName = "100.100.200.40";
                int             PortNumber = 8000;
                snapTime = System.currentTimeMillis();
                try {
                       
<b>permixin = new PersistentMixin ( null , ServerName,
PortNumber</b>
 );
                } catch ( Exception e ) {
                        System.out.println("Unable to open network
connection");
                        System.exit(0);
                }
                long void = permixin.put(ds, fn.getName());
                System.out.println("Time to save object is:"  +
(System.currentTimeMillis() - snapTime));

}

}

Using the calendar to navigate the data sets

The Historical Panel queries the persistent database on initialization of the panel for the existence of saved objects corresponding to days in the current month. If an object exists, the date appears in bold. The subsequent selection of this button by a user will cause the data set corresponding to the date selected to be downloaded as an object. The Java code for that accomplishes this is shown below. The first section of code builds a calendar and associates the object on the server with a

Date

object int the client applet which is then associated with the correct button in the calendar.

class HistoricalPanel extends Panel {
....
for ( int j = 1; j 
c.add(new DateButton(daynumber++ , ServerName, PortNumber ) );
}
add("Center",c);
This section of code queries the server for a DataSet object
for the date selected.
public boolean action(Event evt, Object arg) {
                System.out.println(evt);
                System.out.println(arg);
                PersistentMixin permixin;
//////////////////////////////////////////////////////////////////
// Retrieve the object requested and make it the active data set
/////////////////////////////////////////////////////////////////
                System.out.println("Reading Object");
                try {
                        permixin = new PersistentMixin ( null , ServerName, PortNumber
);
                } catch ( Exception e ) {
                        System.out.println("Unable to open network
connection");
                        return ( true );
                }
                GetResult       result;
              result = permixin.get( ((DateButton)(evt.target)).getKey());
                if ( result.status  > 0 ) {
                        DataSet response = (DataSet)result.o ;
                        app.ds = response;
                        System.out.println("Retrieved "  +
response.toString() + "n");
                        app.startURL = null;
                        app.startConstantUpdate = true;
                        app.heading.setText(((DateButton)(evt.target)).getKey());
                        app.cardsLayout.show(app.cards,"Active");
                        app.active.validate();
                        app.repaint(0);
                } else {
                        System.out.println("Error in transaction: " +
result.status);
                }
                return ( true );
        }
}

VRML display of the data

Several months ago I mentioned a program that produced 3D output that represented HTTP operations for a Web server. While searching for some VRML type interfaces to add to Percollator, I came across an implementation that utilized VRML to create 3D representations of important Web server properties — such as HTTP operations.

Installing Percollator on your own system for development

Percollator requires the following items to be installed on your local system to extend the Percollator applet by adding your own abilities to Percollator. Very simple we need to either generate the data from an existing Web server on your machine using the Percollator.se tool or you need to download some of the URLS that contain the performance data for the JavaWorld servers. Once we have the data we need the Java applets/applications that comprise Percollator. The last item needed is the component library. If you are interested in building your own version of Percollator you can just use the applets with different component libraries. This will take you some time.

• Percollator log file creator programs for Solaris.
• • Introduction to Percollator
  • Advanced Rule Setting
• Percollator source files
• Mixin class libraries used to build Percollator (get the free 30-day evaluation)

Conclusion

Percollator has demonstrated that Java is

robust (a definition of robust a la OOP)

enough to implement performance monitoring programs. During the development of this applet we learned about the following:

• Testing an applet or application on the major browsers is important since there may be differences due to the nasty but required version issues.
• The importance of understanding firewalls and their subsequent limitation on accessing ports outside the well-known range. Considering alternate technologies like IIOP, CORBA, RMI, and JDBC in the design of applets and applications.
• The relevance of new apis like RMI, JDBC and CORBA.
• How to improve the performance of a java application via both programming techniques and the use of JIT compilers.
• The importance of using components to build applets and applications.
• Maintaining state in the user interface
• Using Tabbed Panels to replace the Menu bar approach
• The meaning of persistence and how it helps in the development of applets and applications.
• Using the Mail system to mail dynamic content to users.
• Moving data back and forth between the client and the server.
• Using CGI techniques in Java.
• The importance of the Java Management API.

Futures

The log format developed by Adrian Cockcroft in the Data Collection program would be an ideal standard for all Web servers to generate. Ideally it could just be an SNMP agent that could be queried. Given the source of data and a good selection of components, it is clear that Java provides a superior solution by providing universal access, eliminating the install issue, and being network aware. There are two additional abilities that would be useful:

• Support for Netscape and Internet Explorer in the RMI world
• Realtime 3D representation of the data