Maintaining the LinBox Webservice

On this page I document exactly how the LinBox web service was developed and how it may be updated and maintained. If you have any further questions, feel free to email me at matt.fendt@gmail.com. Please prefix the subject line of your email with [LINBOX].
-- Matthew Fendt, January 2009

The old version of this page can be found here.

Page Contents

Overview
How it Works
Building the Server
Building the Middleman Service and Queue Client
Running the Webservice
Updating and Maintaining the Service
Current Issues and Concerns

Overview

LinBox is a "C++ template library for exact, high-performance linear algebra computation with dense, sparse, and structured matrices over the integers and over finite fields." LinBox can be used to perform massive matrix computations that programs like Matlab and Maple could not solve. Furthermore, LinBox is constantly being updated and improved, with special considerations for potential users.

However, this power comes with a price. It is not always practical or desirable to download the entire LinBox library, as the entire download may not be necessary for only a few computations. Also, since LinBox is constantly in flux, the users would need to stay up to date with the frequent downloads. Finally, since LinBox is a "template library," it is not always easy for the user to run a program and quickly get an answer. This prompted the need for another way of using LinBox.

The creation of a web service for LinBox makes it easy to use the LinBox library. A web service is "a software system designed to support interoperable Machine to Machine interaction over a network" (Wikipedia). The web service solves the above problems with LinBox. First and foremost, the user no longer needs to download the entire LinBox library to perform a computation. He or she simply needs to install the client code that accesses the web service. Second, the web service's copy of LinBox will be able to be updated without any worry from the user. Finally, the web service provides several different methods that the user can simply input the data and receive an answer.

Return to top

How it Works

The web service can be thought of in terms of three clouds:

The first cloud consists of all of the user clients. They are on various machines, and have downloaded the client code that allows them to access the LinBox webservice. These clients make calls to a intermediate webservice called a middleman, which enqueues their requests for processing.
The second cloud consists of two things running on the same machine. The first is the middleman webservice, which is called by the clients. The second is another client, called a "Queue Client," which processes the requests which were added to the queue when the user clients called the middleman webservice. This queue client then calls the actual LinBox webservice.
The final cloud consists of one or more servers on some number of machines. These servers host the actual LinBox webservice that the queue client calls. The answer is computed and given back to the queue client, which then gives the answer back to the user client.

The purpose of the extra intermediate webservice and the queue is so that users can asynchronously call the webservice, and their requests can be processed in an orderly manner. If the user was allowed to call the webservice directly (which is how the webform works at this time), no other user could call the service until the first call was finished processing.

Return to top

Building the Server

A "server" is the computer on which the web service will be hosted. It is also where the LinBox library is installed and where the computations are done. The following contains the step that were taken to install the server on hmrg.

Some things to note:

Currently only configured for Fendt's directory on "hmrg.pc.cis.udel.edu"
Web service resides in /home/fendt/apache-tomcat-6.0.18/webapps/ axis2/WEB-INF/services
Apache is currently listening on port 2000. This is verified in /WEB-INF/server.xml The "Connector port" is set to 2000. This is at line ~50.
Apache startup/shutdown locations:
STARTUP: /home/fendt/apache-tomcat-6.0.18/bin/startup.sh
SHUTDOWN: /home/fendt/apache-tomcat-6.0.18/bin/shutdown.sh

Requirements:

Apache Ant 1.7.0
Apache Tomcat 6.0.18
Axis2 1.4.1 installed under Tomcat
JDK version 1.6.0_07
SWIG version 1.3.31 (For updating LinBox functionality)
LinBoxWebservice directory
LinBox computational library itself

Setup:

Install Ant, Tomcat, Axis2, and Java
Set CATALINA_HOME to Tomcat directory (eg /home/fendt/apache-tomcat-6.0.18)
Set AXIS2_HOME to Axis 2 directory (eg /home/fendt/axis2-1.4.1)
Set ANT_HOME to Ant directory (eg /home/fendt/apache-ant-1.7.0)
Set JAVA_HOME to Java directory (eg /usr/lib/jvm/java-6-sun)
Make sure Apache is shut down with /home/fendt/apache-tomcat-6.0.18/bin/shutdown.sh
Svn and configure the LinBox library. This was done on hmrg.pc.cis. udel.edu in the directory /home/fendt/linbox. This is the copy of LinBox that the web service will use.
Copy the LinBoxWebservice directory, located at /home/fendt/LinBoxWebservice.
In LinBoxWebservice directory, type "ant generate.service" to ready the service.
Start up Apache with ./home/fendt/apache-tomcat-6.0.18/bin/startup.sh
The web service is now installed in the "services" directory on Apache, is listening on port 2000, and ready for client calls.

Return to top

Building the Middleman Service and the Queue Client

Once the service is installed, you will need to generate the intermediate webservice and the queue client.

Make sure that Apache is started up on hmrg.
Generate the middleman web service on linalg with ant generate.middleman
Generate the queue client on linalg with ant generate.queue.client
The queue client is now ready to be run.

Return to top

Running the Webservice

Once the service has been installed, you can choose to run either the stand alone client or the dependent client. To install and learn more about these clients, click here.

To run the dependent client:

Make sure the service is installed.
Start up Apache.
The dependent client can now be called.

To run the stand alone client:

Make sure that the service and middleman have been generated, as above.
Start up RMI on linalg with /usr/jdk/jdk1.5.0_15/bin/rmiregistry
Start the queue client on linalg with ant run.queue.client
You can now make web service calls on hmrg with the user client.

Return to top

Updating and Maintaining the Service

Compilation of LinBox Functional .so From Default Directory on Hmrg

cd to /home/fendt/LinBoxWebservice/src/samples/quickstart/service/adb/xsd
In makefile: Make sure that the path of the LinBox library is correct. It is currently -I/home/fendt/linbox
Type "make liblinboxfunctions.so"
The .so should be ready to be used.

Relocating the LinBox Function .so

In makefile, set the path to the LinBox library, eg "-I/home/fendt/linbox"
Follow compilation instructions as above

To Add Additional LinBox Functionality

In linboxfunctions.i: Add both the computation function, eg
- extern bool det(std::istream& matrix_in, std::ostream& det_out)
and the i/o function, eg
- extern char* detFiles(char *matfile)
both inside and outside the %{ ... %} block
In linboxfunctions.C: Update the "Currently supported methods" section, #include the appropriate files from the linbox library, and add the appropriate "file" variable. Add new computation and i/o function.
Run the .i file through SWIG (swig -java -c++ -package samples.quickstart.service.adb.xsd linboxfunctions.i). This will generate the necessary interfacing for the C++ linbox code to work with the Java web service.
Type "make liblinboxfunctions.so"
The .so should be ready to be used. Note that if only "linboxfunctions.C" needs to be changed, step 3 can be ommited.
The resources/META-INF/services.xml file needs to be updated; add a new "operation" tag to the file.
The service should be reinstalled on hmrg so that the new service is added.
The client code also needs to be updated to accomidate the new functionality.

Return to top

Current Issues and Concerns

At the end of the summer, the hmrg computer was updated to Ubuntu, and this caused some things to stop working in the service. During this winter, I fixed these issues, and both the queued command line server and the web client are running.
Axis2 was upgraded to version 1.4. This helped with the issue above, but introduced a new warning:
[WARN] Unable to generate EPR for the transport : http
This is seen during the wsdl generation of the middleman and TransferAgent services. It is uncertain that this warning has any effect.
A warning:
[java] log4j:WARN No appenders could be found for logger (org.apache.axis2.description.AxisService).
[java] log4j:WARN Please initialize the log4j system properly.
occured when running the queue and user clients. This was solved by adding the directory containing a log4j file to the runtime directory in the Ant file.
The TransferAgentMiddleman web service and the queue client were moved over to linalg.org from hmrg. Everything still works.

Return to top

Return to server page