Monday, March 30, 2009

Xen: finishing the job

Once upon a time, Xen was the hot virtualization story. The Xen developers had a working solution for Linux - using free software - well ahead of anybody else, and Xen looked like the future of virtualization on Linux. Much venture capital chased after that story, and distributors raced to be the first to offer Xen-based virtualization. But, along the way, Xen seemed to get lost. The XenSource developers often showed little interest in getting their code into the mainline, and attempts by others to get that job done ran into no end of obstacles. So Xen stayed out of the mainline for years; the first public Xen release happened in 2003, but the core Xen code was only merged for 2.6.23 in October, 2007.

A nice article by Jonathan Corbet, which compare Xen and KVM. (worth reading)

Thursday, March 26, 2009

Apache Software Foundation is 10 years old

Apache Software Foundation celebrated its 10 year anniversary at ApacheCon EU. According to my knowledge ASF is one of the best open source foundation in the world. And it has a number of different projects. I am so happy to be a part of ASF and I am contributing to apache since 2004, so it is almost 5 years now. ASF has creatde a web page including most of the highlights of the last 10 years, I think it is worh reading.

Wednesday, March 25, 2009

Amazon EC2 – Building Apache Web Server

In my previous post I explained how to do ssh and scp among EC2 instances, now I am going to discuss how to install Apache on EC2 (Fedora8). I know building Apache is not such a difficult work if you have setup the environment with all the essential libraries. However if it is not then it is some what difficult, in this post I am going to explain some of the challenges you may encounter if you try to build Apache on Fedora 8 (on EC2). You might imagine what is the difference between normal Fedora and Fedora for EC2, the main difference is EC2 runs on Xen hypervisor, so what you have is para-virtualization (you modify the OS to run on Xen).

First step is to install all the required libraries. So you need to install gcc and g++,
Installing gcc : yum install gcc
Installing xmms : yum install xmms
Installing g++ : yum install gcc-c++

In my case doing all those did not allow me to build the Apache, it was giving me an error saying “asm/page.h” does not found. Then I realize some of the OS related header files are missing in the OS image. To solve that I had to copy “/usr/include/asm” from a normal Fedora OS. After I did all those, boom Apache worked fine.

Just follow the same procedure if you want to setup your own dedicated servers.

Sunday, March 22, 2009

Amazon EC2 – Communicating amoung instances

These days I am doing some cool work with Amazon EC2, so there were a lot to learn and a lot to find out. One of the major challenge I faced was to communicating between instances. For my application I need to do “ssh” and “scp” among the nodes, and most of those taking place automatically. So I had hard time figuring out this, the main reason is in EC2 if you want to connect to an instance you have to give your key-pair. Which is not possible for my application. So what I did was following.
  • Run an instance with the your operating system (in my case I choose Fedora 8), when you run select the key-pair and security group.
  • Next you need to generate SSH key, you can do that using following command (Do not enter a passphrase)
  • ssh-keygen -t rsa
  • Now you need to go and edit the ~/.ssh/authorized_keys , For that copy the content of .ssh/id_rsa.pub and paste
  • Now open the ~/.ssh/authorized_keys, and change the tail to be “root@domU” , basically you need to remove root@ec2-.. with new one
  • Then you need to edit the “~/.ssh/know_hosts” file and add the following,
  • *.amazon.com ssh-rsa Asasdas....
  • (Asasdas.... = content of rsa.pub)

Now you are all set (unless you need to install some other packages), next step is to bundle your image and register that with EC2 and S3. Then try running more than one instances, now if you try to do ssh among those you will find that you do not need to type the password, or you do not need to specify the key-pairs.

Wednesday, March 11, 2009

AWS Management Console

If you are using Amazon EC2, then you would find the new AWS Management Console as a very useful tool. I agree you can have the same functionality using normal SOAP API but this is cool. As I see so far it is just read only data, but provide a handy way to look at the instances you have, AMIs you have created, various secuirty gropus, kep-pairs and etc.., I think it should provide additional methods such as,
- terminating an instance
- Deleting an AMI
- And may be remove data from S3

Then you can do most of the work without worring about the command line tools.