Wednesday, October 29, 2008

VMWare Recording and Replay

Simply, it allows for making a recording of (almost) everything that happens to a VM between the time you hit Record and the time you hit Stop. This is not a movie recording, but more of an execution recording. You can play it back however many times you like. Most important thing is it records at the instruction level , so it keeping the complete memory and system state.
The process is said to be deterministic , because it keeps all the instruction that send from the guest operating system. Which include all the IO instructions (Network , CD and all kind of IO) , user clicks and any other type of asynchronous call as well.

What is this good for? Well, have you ever tried testing a program only to encounter a bug that you just can’t reproduce? Maybe there was some memory corruption that happened under some specific case that you just can’t seem to diagnose. Or maybe it’s a network packet that came in in some form that your application didn’t expect. Under normal circumstances, you’d have to do a lot of guesswork in order to find out what exactly happened. Far too often, it’s just too hard to reproduce the bug and it goes unfixed for some time.

Now imagine instead that you’re testing the program in Workstation and, before your testing, you hit Record. You attempt the test and the program crashes in some weird manner. No problem. Hit Stop and replay the recording. Just before the crash occurs, stop the playback and attach a debugger. Messed up? Didn’t find the cause? Replay that recording again.

In addition to the programming debugging this can be used as a fault tolerance and clustering technique. Where you can create an environment with same VM image and then you can pick one of them as the primary and other one as the backup . When the actual invocation happen rather then recording the events in the primary image , you can send them to the backup image. So if the primary fails , backup can easily take over. Because it is not just a replicate , it is an identical to the primary at anytime.

Finally I consider this as a very handy tool as well as can be used to many application to save the time. May be in the future when you report a bug you can attach the VM record log so that other people can easily reproduce the error.

Thursday, October 16, 2008

Yahoo is going open ?

Today I got a chance to listen to a very interesting talk from Dr. Larry Heck , he is the Vice President, Search & Advertising Sciences, Yahoo. The title of the talk was Large Scale Data Analysis for Web Search & Online Advertising R&D Using Pig™ and Hadoop™ . The talk is interesting due to few reasons , first he did a great job explaining how yahoo search and yahoo advertisement works. And he explained some of the algorithm behind the secret of yahoo search and advertisement. One thing I heard most useful is that seems like Yahoo is moving towards open source ,and they are try to open all of there search API open so that would be a very good news. Actuallty as the frist setp that they have open sourced and controbuting to Apache Hadoop and Apache Pig. It is amaze to see the perfromance improvements they have gained after they have moved to above metiuoned apache tool. They have able to reduce some of the taksed which took more than few days to less than hour.

Part of the abstarct from his talk

Pig is an open-source platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig™ programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets. Its infrastructure layer consists of a compiler that produces sequences of Map-Reduce programs, for which large-scale parallel implementations already exist (e.g., Hadoop).

Tuesday, October 14, 2008

Axis2 tutorial at Apachecon US 2008

I am planing to do an Axis2 tutorial in ApacheCon US 2008 , so if you are interested in attending that tutorial I think its time for registration. The tutorial will cover , introduction to Axis2 , Deploying Axis2 , Writing services , invoking services , session management and etc..

Monday, October 13, 2008

How to Control Binding Generation in Axis2 ?

Whenever you deploy a service in Axis2 , it generates a WSDL for you. You are able to see the WSDL generated when by executing a “?wsdl” for your service. If you look closer you will see that the WSDL generated has three bindings.

* SOAP11 binding
* SOAP12binding
* HTTPbinding

So next question is how to control the binding generation behavior. To address that I wrote a knowledge base in OT. Read full KB

Sunday, October 12, 2008

Good lesson not to trust electronic equipment

Today something interesting thing happened to me. I had to drop one of my friend into his place. Actually he bought a new car recently and I had to help him to learn how to drive and then drop him to where he live. So we got the printout of the direction, however some other friend of mine gave me a GPS navigator. So I thought that is easier than reading printout and go.

So I put the destination on the GPS navigator and asked to select the root skipping highways. So it did a good job and directed me how to go. So I was driving for a while , may be about 15-18 miles, I took few left and right turn. So now I have no idea where am I , and actually I do not need to know. Because GPS can help me to go there. Unfortunately when I was driving I suddenly realized that the GPS does not work !!!! , I got STUCK

So now I am in a big trouble I have no idea where I am , and I do not have a city map as well. So I took few roots to see whether I find any of the familiar road, unfortunately I did not find any. So I decided to go to a gas station and ask about that. So I went there and ask the person about the location I need to go , he said no he do not know. Then I asked him whether he knows how to get in to Georgia Tech , then he said yes. So luckily he told me the to take the high way , but he said that he is not sure about the exist. So I took the highway and went about few miles, oh I found an exist which I know !!!

So it was a good lesson for me ,I should not always trust the electronic items , we need to have something ready when we faces some unexpected situation like this.

Friday, October 10, 2008

Embedding Apache Axis2 into Existing Applications

Axis2 can be deployed and used in various manner , we can use Axis2 as a client to invoke a remote service , we can use Axis2 as a Web services server to deploy Web services. And we can deploy Axis2 inside an application server and deploy Web services. In addition that we can embed Axis2 into existing application , so I wrote an article to discuss the procedure of embedding Axis2 into an existing application.

Read the full story

Saturday, October 04, 2008

Request Arrival time tracking with Tomcat

This is something I really wanted but I did not find that so after doing some amount of work I found a way to track the request arrival time of Tomcat. By default it records request depart time and service time. There are some instances having that is not enough , so I did the following and get the job done.

- Implement the lifecycle interface that Tomcat has
- Then add a Value component in Tomcat (server.xml)

So class would be something like below , of course it is very simple one , need to modified but it just does the job.

public class TimeTracker
extends ValveBase
implements Lifecycle {

protected LifecycleSupport lifecycle = new LifecycleSupport(this);
private boolean started = false;

public ResourceTrackingAccessLogValve() {
super();
}


public void invoke(Request request, Response response)
throws IOException, ServletException {

// Pass this request on to the next valve in our pipeline
long t1 = System.nanoTime();
getNext().invoke(request, response);
long t2 = System.nanoTime();
long time = t2 - t1;
System.out.println("ArriveTime ns " + t1 + " Departtime ns " + t2 + " ServiceTime : " + time);

}

public void addLifecycleListener(LifecycleListener listener) {

lifecycle.addLifecycleListener(listener);

}


public LifecycleListener[] findLifecycleListeners() {

return lifecycle.findLifecycleListeners();

}


public void removeLifecycleListener(LifecycleListener listener) {

lifecycle.removeLifecycleListener(listener);

}


public void start() throws LifecycleException {
// Validate and update our current component state
if (started)
throw new LifecycleException
(sm.getString("accessLogValve.alreadyStarted"));
lifecycle.fireLifecycleEvent(START_EVENT, null);
started = true;
}


public void stop() throws LifecycleException {

// Validate and update our current component state
if (!started)
throw new LifecycleException
(sm.getString("accessLogValve.notStarted"));
lifecycle.fireLifecycleEvent(STOP_EVENT, null);
started = false;

}


}


And then need to add the following Value tag in server.xml

<Valve className="org.log.ResourceTrackingAccessLogValve"
directory="logs" prefix="localhost_resource_log."
suffix=".txt"
domain="Catalina"
pattern="%a %t %D %s %b %U %q %m" resolveHosts="false"/>