Pradeep Padala's blog
Virtually the best blog on the web!
Challenges in Mobile (Smart Phone) Virtualization
Feb 12th
We have virtualization on servers, desktops, laptops and what’s next? Naturally, the “smart phone”. Today’s mobile phones have become quite powerful that one can easily virtualize them and run multiple operating systems. But,
Why do you need smart phone virtualization?
Similar to desktop virtualization, the use-case is not that clear, but there are some benefits that can be gained.
- Reducing vendor lock-in. Can you run iPhone applications on an Android phone? Currently, no, but with virtualization this might be possible. There are issues with Apple (or other vendors)’s mobile OS licensing that make it illegal to do so, right now. You can only run iOS on Apple approved devices, but this may change in the future. Many old phones have legacy applications and libraries that can be run in virtualized sandboxes as well.
- Mobile cloud computing. You can clearly access the so-called “cloud” from a smart phone, but if smart phones are virtualized, mobile applications can be migrated to a cloud and back seamlessly. This is still a pipe-dream and a few research platforms (MAUI and CloneCloud) are proposed to make this a reality. The benefits of doing this are enormous: one can potentially access infinite compute and storage resources on the smart phone.
- Security. This is probably the strongest argument for mobile virtualization. The popularity of smart phones is making them a prime target for malicious hackers. Malware, Viruses and other nasty stuff are making their way into smart phones, which currently don’t have much protection against them. It’s not easy to run Anti-virus software on a mobile phone, as the current AV software is very resource hungry making it unsuitable for smart phones.
- Isolation. This is related to security. It will be possible to have different user profiles with different levels of access (say work and home) with mobile virtualization. VMware recently demonstrated, how multiple user profiles can co-exist in the same phone at Mobile Congress 2011.
- Ease of development. This is an indirect benefit for mobile phone developers. Virtualization can make the development of mobile applications easier by removing the need for maintaining multiple versions of the same application for different operating systems.
Challenges
Before you start seeing virtualized mobile phones, there are still plenty of challenges that need to be solved.
- Limited resources. Though, the CPUs on mobile phones are getting more and more powerful, they still have limited battery and storage space. Virtualization usually adds more burden to the resources and is not necessarily energy-aware.
- Poor connectivity . Mobile cloud computing sounds like a great idea, but mobile phones often experience variable connectivity. It’s not easy to implement disconnected operations for storage and for seamlessly migrating computation.
- Poor performance. Virtualization adds performance overheads, which are still being researched in server virtualization environments. It’s not clear whether the overheads are worth the effort on a mobile phone.
These challenges can be overcome and it’s only a matter of time before smart phones are virtualized. Some current players in the market are: VMware’s mobile virtualizaiton platform (MVP) and OKL4 MicroVisor. OKL4 claims to be the first open source mobile hypervisor, which is an exciting development that can lead to advances in mobile virtualization.
How to Access Your Web Server within Your Local Network
Feb 9th
If you are hosting a web server at home and using your router’s port forwarding to redirect requests from WAN port to WebServerHost:80, there’s an annoying problem in some routers: you cannot access the web server from inside the network. There are multiple reasons why this might happen.
- The router assumes that all traffic from internal hosts to port 80 are for the router admin interface. This is annoying, as you just get to the router’s web admin page. Solution: you can port forward from another port (say 8000) to your WebServer’s port 80. Now, you can access the website with http://my.site.net:8000. You are out of luck, if your router does’t allow port forwarding to a different port (say WAN:8000->WebServerHost80 instead of WAN:80->WebServerHost80).
- You have two or more routers with different subnets and the routing is messed up. This is a difficult problem to diagnose and often the router’s admin interfaces are not that amenable to debugging routing problems. I suggest changing your firmware to DD-WRT, if you can.
- Port forwarding doesn’t work for incoming traffic from local ports on the router. I think this is probably the most common reason. Without DD-WRT, there’s no easy way to fix this. Solution: One simple way to solve this is to set a hard-coded DNS entry in
/etc/hosts(Linux) orc:\WINDOWS\system32\drivers\etc(Windows) as follows.192.168.x.x my.site.net
192.168.x.x is the internal IP address of the WebServerHost
Computer Science Conferences – Statistics and Acceptance Rates
Feb 9th
There are a bunch of sites with statistics on computer science conferences, but it always bothered me that there was no central place to search for conference statistics like acceptance rates. I also wanted to have a visual display of statistics similar to Google Finance. In this age of AJAX and Web 2.0, the web page should also be very fluid.
My first attempt to build such a web page is here. Right now, it scrapes the data from Kevin Almeroth’s excellent stats page and presents it in a visual form. The scraping is still a bit rough, so there might be some errors. I would like to add more features like comparing conferences, searching and adding more conferences.
Leave your comments and suggestions below.
Best Resources to Learn about Linux Kernel Internals
Feb 7th
The Source
The best resource is the kernel source.
- I prefer reading the cross-referenced lxr, which makes it easy to follow the code.
- Linux Networking Subsystem: Desktop Companion to the Linux Source Code. A commentary Linux networking code.
- The Linux Kernel & the File System Subsystem : An Architectural Overview. Talks about VFS in the Linux.
- Code Commentary On The Linux Virtual Memory Manager by Mel Gorman.
Books
Obviously, it’s not that easy to dive into thousands of lines of code. I suggest starting with reading the books explaining Linux kernel in general.
- Understanding Linux Kernel. This is one of the first books to provide in-depth explanations. Vastly improved over multiple editions, the current one is a very good read.
- Robert Loves’ book. Love is a core developer, who implemented pre-emptive kernel and other features. I haven’t personally read this, but I have seen good reviews of this.
- If you are specifically looking for networking aspects, this is an excellent book on understanding linux networking internals.
- For device drivers, similarly, this book is very useful.
Other resources
- LWN’s kernel page has lots of great articles explaining kernel internals
- TLDP’s TLK. Somewhat outdated, but useful.
- Linux Journal’s KernelKorner has some excellent articles, most of which are freely available online.
- The Linux Kernel Hackers’ Guide, compiled by Michael K. Johnson of Red Hat fame. Includes among other
documents selected Q/A’s from the linux-kernel mailing list.
HOWTOs
- The LinuxKernel HOWTO by Brian Ward.
- A completely new Kernelhacking-HOWTO at http://www.kernelhacking.org/.
- Various Kernel HOWTOs on specific questions, such as the BogoMips mini-HOWTO by Wim van Dorst.
- Various Linux HOWTOs at TLDP
Lists of links
- There’s a huge index of links at http://jungla.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html that has links to many resources.
- Another list of links maintained by Chris Gould. Mostly unsorted
Virtualization Bibliography
Feb 2nd
Virtualization is a hot topic and the field is growing at a rapid pace. There are hundreds of papers on each sub-topic, and this list is intended to be a good starting point for someone starting in virtualization research.
Surveys/Books
These are good starting points, if you are just learning about virtualization
- Survey of virtualization machine research. Robert P. Goldberg. IEEE Computer, June 1974, pp 34-45. [PDF]. One of the oldest surveys about virtualization research.
- A Survey on Virtualization Techniques. Susanta Nanda and Tzi-cker Chiueh [PDF]. A more modern survey of virtualization techniques. The paper lists many of the techniques, but doesn’t really explain how they are different etc. Yet, this is a good read for budding virtualization researchers.
- Virtual Machines: Versatile Platforms for Systems and Processes . Jim Smith Ravi Nair. Amazon book link
Virtualization Overview
These are the papers that will help in understanding fundamentals and concepts of virtualization.
- Xen and the art of virtualization. Barham et.al. This is the classic SOSP paper on para-virtualization. [PDF].
- When Virtual is Better Than Real. Peter Chen and Brian Noble. This is a great short article explaining the benefits of virtualization. ACM link.
- Disco: Running Commodity Operating Systems on Scalable Multiprocessors. E. Bugnion, S. Devine, and M. Rosenblum. This paper is considered to be the first paper that revived the virtualization concepts pioneered by IBM. ACM link.
- Running multiple operating systems concurrently on an IA32 PC using virtualization techniques. Kevin Lawton. This is a great article on the difficulties involved in virtualizing the x86 platform. link.
- Formal requirements for virtualizable third generation architectures. Popek and Goldberg. The classic paper explaining the requirements for virtualizing a specific ISA. ACM link. If you want light reading, check the Wikipedia explanation of the requirements.
- Scale and Performance in the Denali Isolation Kernel. Andrew Whitaker, Marianne Shaw, and Steven D. Gribble. Denali is another example of early virtualization papers that rejuvenated the research. Denali shows the containers (or vservers) concept, which is used in OpenVZ, KVM and VServers.
- The Exokernel Operating System Architecture. Dawson Engler’s thesis. [PS]. This is more of OS research, but a great read for understanding some of the techniques (thin hypervisor, pass through etc.) that are used in modern virtualized systems.
CPU Virtualization
Virtualization of CPU is provided by a CPU scheduler that provides the illusion of multiple CPUs (or VCPUs). Scheduling has a long and rich history. Below are a few links in relation to virtualization.
- Xen’s credit scheduler. Xen’s credit scheduler is a proportional fair scheduler, which is pretty big improvement (especially in SMP environments) over the old SEDF scheduler.
- Xen’s scheduler is partly inspired by Carl Waldspurger’s lottery scheduling and the paper: Lottery Scheduling: Flexible Proportional-Share Resource Management by Carl Waldspurger is a must read. [PDF].
- Comparison of the Three CPU Schedulers in Xen – Ludmila Cherkasova, Diwaker Gupta, Amin Vahdat – Perfomance Evaluation Review. Vol 35, Number 2. Sept 3, 2007. This provides a great comparison of CPU schedulers and provides insight on how handling I/O is complicated. [PDF].
Network Virtualization
Network virtualization is gaining momentum in recent years. I would just point you to the following survey as a starting point.
- Survey of network virtualization. N.M. Mosharaf Kabir Chowdhury and Raouf Boutaba. [PDF]. This is pretty comprehensive and well-written survey on network virtualization. Mosharaf also has a nice network virtualization, listing lots of useful papers.
Device Virtualization
- Xen’s device driver virtualization.
- Virtualizing I/O Devices on VMware Workstation’s Hosted Virtual Machine Monitor. Jeremy Sugerman, Ganesh Venkitachalam, and Beng-Hong Lim. As the name suggests, this paper explains VMware Workstation’s mechanisms for virtualizing devices. [PDF].
- GPU Virtualization on VMware’s Hosted I/O Architecture. Micah Dowty, Jeremy Sugerman. This paper by VMware folks explains how VMware implements GPU virtualization. Great starting point for GPU virtualization. [PDF]. Andres’ paper on VMM indepedent graphics acceleration is a good follow-up reading.
Memory Virtualization
- Memory Resource Management in VMware ESX Server. Carl A. Waldspurger. [PDF]. Classic paper on memory virtualization. This paper introduces memory ballooning, which is a great technique used in commercial platforms as well.
- Difference engine: harnessing memory redundancy in virtual machines. Diwakar Gupta et.al. [PDF]. A great follow up paper that talks about sharing memory across VMs. Incidentally, both these papers have received best paper awards.
Migration/Cloning
- Live Migration of Virtual Machines. Clark et.al. One of the first academic papers discussing how to do live migration, which is implemented in Xen. VMware has supposedly implemented vmotion before this was published. [PDF]
- The design and implementation of Zap: a system for migrating computing environments. Osman et.al. Zap predates Xen paper and talks about sandboxing processes that can be migrated. This is more like container virtualization rather than full system virtualization. ACM link
- SnowFlock: rapid virtual machine cloning for cloud computing. Andres Lagar-Cavilla et.al. Greatp paper to read to understanding cloning of virtual machines. Snowflock link.
Resource Management/Automation
- Carl Waldspurger’s PhD thesis is a great place to start for understanding resource management. MIT link.
- Automated control of multiple virtualized resources. Padala et.al. Disclaimer: this is my own paper
It’s a good place to start if you want to learn about automating resource management for virtual machines. The control theory aspects are a bit complex, but you can ignore them and focus on the issues in automating resource management. [PDF]. - Black-box and Gray-box Strategies for Virtual Machine Migration. Clark et.al. This is a follow up for live migration paper. The paper talks about strategies for automating the migration of VMs to meet specific goals. USENIX link.
How to Convert PowerPoint Images to PDF
Jan 13th
I am doing this a lot more often these days, want to convert the pretty images I created with PowerPoint into PDF, so that they are more amenable to Latex/PdfLatex. For fun, I prepared an illustrated HOWTO. Here you go, should be easy to follow.
Let’s say you have a slide with following picture.
First, you want to save the slide as PDF.
Don’t save it yet. Choose options to make sure that you are selecting only one slide.
As soon as you save it, Adobe Acrobat will open it. You may want to remove all the extra white space around it. To crop the image, you will need the Acrobat Writer (Reader is not enough to do the following task). Choose the crop tool, as shown below.
Choose the region you want to crop, and double click inside the region. You will see a dialog similar to the one shown below.
Press ok, and you are done.
Have fun!
P.S. On Mac, you don’t have any hassle. Just select the region you want to put in image. Right click and save as picture (choose PDF as the file format).
2011 Resolutions
Jan 6th
I wanted to write a detailed post about how 2010 went, but it seems like my 2011 is already in full swing. 2010 was amazing and one of the best years in my career. Highlights include my best paper award and the GENI high profile demo. I also started running and lost about 5 kilos.
So, what’s in store for 2011?
- Focus. I worked on a lot of different things in 2010, but in 2011, I hope to focus on one specific project (yet to see the light of publication) and pool all my resources into it.
- Writing. I have a ton of back log on writing research publications and blogging. Hope to change that this year. Not sure, if I can get to blog more often. I really enjoy writing the HOWTOs (1, 2) that seem to get a lot of hits. Hope to do more of those.
- Running. I probably ran about 250-300 miles this year (started in May and skipped two months in between), which is ok, but not great. My goal in 2011 is to run at least 700 miles and hope to lose 5-10 kilos. Will try more difficult terrains like dish run.
That’s it! Happy New Year!
How to Build Custom Xen Kernel Based on CentOS/RHEL Xen Kernel
Oct 1st
The HOWTO on compiling CentOS kernel based on RHEL code base (currently 2.6.18) is overly complicated. Follow the following steps to build a custom Xen kernel, maybe with different kernel configuration.
- Setup the rpmbuild directory, and install prerequisite packages.
mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS} sudo yum -y install rpm-build redhat-rpm-config unifdef kernel-devel kernel-xen-devel - Grab the kernel source RPM, you want to modify and compile, and install it.
wget -c http://mirrors.kernel.org/centos/5/updates/SRPMS/kernel-`uname -r`.src.rpm rpm -i kernel-`uname -r`.src.rpm
- Create the spec file for the downloaded kernel
cd ~/rpmbuild/SPECS rpmbuild -bp --target=`uname -m` kernel-2.6.spec
- Modify the config to your needs. The config file are stored in
~/rpmbuild/SOURCES. - Modify the buildid. This is a good practice, if you are making extensive changes.
cd ~/rpmbuild/SPECS sed -ibak -e 's/#% define buildid/%define buildid .ppadala/' kernel-2.6.spec
- Build the kernel
cd ~/rpmbuild/SPECS sudo rpmbuild -bb --with xenonly --without debug --without debuginfo --target=`uname -m` kernel-2.6.spec
- This will generate a RPM that you can install on other machines as well.
Using kpartx to Mount VM Disk Images
Sep 29th
If you are using a full disk image for your VM, instead of using a file system, you cannot simply mount it directly using
mount -o loop d0.img
Use the following commands to see the partitions in your disk, and then mount them.
# kpartx -av d0.img add map loop0p1 : 0 29333504 linear /dev/loop0 2048 add map loop0p5 : 0 1380352 linear /dev/loop0 29337600 # mount /dev/mapper/loop0p1 /mnt # ls /mnt bin cdrom etc initrd.img lib media opt root selinux sys usr vmlinuz boot dev home initrd.img.old lost+found mnt proc sbin srv tmp var # unmount /mnt # kpartx -d d0.img
The last two commands are to unmount and delete the partition mappings.
How to Setup Xen4 and pvops 2.6.32.x on CentOS 5.5 using GITCO repository
Sep 28th
See my howto on setting up Xen4 + pvops 2.6.32.x first before following this.
- Do steps 1, 2 and 3 described in the howto
- Install Xen 4.0.1 from GITCO repository. The benefit of this is that all the required packages,
Xen enabled Linux kernel (2.6.18) and libvirt are correctly installed and setupwget http://www.gitco.de/repo/GITCO-XEN4.0.1_testing_x86_64.repo yum -y install xen
- Get pvops kernel
git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen cd linux-2.6-xen git checkout -b xen/stable-2.6.32.x origin/xen/stable-2.6.32.x
- Build the kernel to match your configuration
make menuconfig
Follow the remaining steps in the howto to setup grub, and test it.