Archive for February, 2011
We have virtualization on servers, desktops, laptops and what’s next? Naturally, the “smart phone”. Today’s mobile phones have become quite powerful that one can easily virtualize them and run multiple operating systems. But,
Why do you need smart phone virtualization?
Similar to desktop virtualization, the use-case is not that clear, but there are some benefits that can be gained.
- Reducing vendor lock-in. Can you run iPhone applications on an Android phone? Currently, no, but with virtualization this might be possible. There are issues with Apple (or other vendors)’s mobile OS licensing that make it illegal to do so, right now. You can only run iOS on Apple approved devices, but this may change in the future. Many old phones have legacy applications and libraries that can be run in virtualized sandboxes as well.
- Mobile cloud computing. You can clearly access the so-called “cloud” from a smart phone, but if smart phones are virtualized, mobile applications can be migrated to a cloud and back seamlessly. This is still a pipe-dream and a few research platforms (MAUI and CloneCloud) are proposed to make this a reality. The benefits of doing this are enormous: one can potentially access infinite compute and storage resources on the smart phone.
- Security. This is probably the strongest argument for mobile virtualization. The popularity of smart phones is making them a prime target for malicious hackers. Malware, Viruses and other nasty stuff are making their way into smart phones, which currently don’t have much protection against them. It’s not easy to run Anti-virus software on a mobile phone, as the current AV software is very resource hungry making it unsuitable for smart phones.
- Isolation. This is related to security. It will be possible to have different user profiles with different levels of access (say work and home) with mobile virtualization. VMware recently demonstrated, how multiple user profiles can co-exist in the same phone at Mobile Congress 2011.
- Ease of development. This is an indirect benefit for mobile phone developers. Virtualization can make the development of mobile applications easier by removing the need for maintaining multiple versions of the same application for different operating systems.
Before you start seeing virtualized mobile phones, there are still plenty of challenges that need to be solved.
- Limited resources. Though, the CPUs on mobile phones are getting more and more powerful, they still have limited battery and storage space. Virtualization usually adds more burden to the resources and is not necessarily energy-aware.
- Poor connectivity . Mobile cloud computing sounds like a great idea, but mobile phones often experience variable connectivity. It’s not easy to implement disconnected operations for storage and for seamlessly migrating computation.
- Poor performance. Virtualization adds performance overheads, which are still being researched in server virtualization environments. It’s not clear whether the overheads are worth the effort on a mobile phone.
These challenges can be overcome and it’s only a matter of time before smart phones are virtualized. Some current players in the market are: VMware’s mobile virtualizaiton platform (MVP) and OKL4 MicroVisor. OKL4 claims to be the first open source mobile hypervisor, which is an exciting development that can lead to advances in mobile virtualization.
If you are hosting a web server at home and using your router’s port forwarding to redirect requests from WAN port to WebServerHost:80, there’s an annoying problem in some routers: you cannot access the web server from inside the network. There are multiple reasons why this might happen.
- The router assumes that all traffic from internal hosts to port 80 are for the router admin interface. This is annoying, as you just get to the router’s web admin page. Solution: you can port forward from another port (say 8000) to your WebServer’s port 80. Now, you can access the website with http://my.site.net:8000. You are out of luck, if your router does’t allow port forwarding to a different port (say WAN:8000->WebServerHost80 instead of WAN:80->WebServerHost80).
- You have two or more routers with different subnets and the routing is messed up. This is a difficult problem to diagnose and often the router’s admin interfaces are not that amenable to debugging routing problems. I suggest changing your firmware to DD-WRT, if you can.
- Port forwarding doesn’t work for incoming traffic from local ports on the router. I think this is probably the most common reason. Without DD-WRT, there’s no easy way to fix this. Solution: One simple way to solve this is to set a hard-coded DNS entry in
c:\WINDOWS\system32\drivers\etc(Windows) as follows.
192.168.x.x is the internal IP address of the WebServerHost
There are a bunch of sites with statistics on computer science conferences, but it always bothered me that there was no central place to search for conference statistics like acceptance rates. I also wanted to have a visual display of statistics similar to Google Finance. In this age of AJAX and Web 2.0, the web page should also be very fluid.
My first attempt to build such a web page is here. Right now, it scrapes the data from Kevin Almeroth’s excellent stats page and presents it in a visual form. The scraping is still a bit rough, so there might be some errors. I would like to add more features like comparing conferences, searching and adding more conferences.
Leave your comments and suggestions below.
The best resource is the kernel source.
- I prefer reading the cross-referenced lxr, which makes it easy to follow the code.
- Linux Networking Subsystem: Desktop Companion to the Linux Source Code. A commentary Linux networking code.
- The Linux Kernel & the File System Subsystem : An Architectural Overview. Talks about VFS in the Linux.
- Code Commentary On The Linux Virtual Memory Manager by Mel Gorman.
Obviously, it’s not that easy to dive into thousands of lines of code. I suggest starting with reading the books explaining Linux kernel in general.
- Understanding Linux Kernel. This is one of the first books to provide in-depth explanations. Vastly improved over multiple editions, the current one is a very good read.
- Robert Loves’ book. Love is a core developer, who implemented pre-emptive kernel and other features. I haven’t personally read this, but I have seen good reviews of this.
- If you are specifically looking for networking aspects, this is an excellent book on understanding linux networking internals.
- For device drivers, similarly, this book is very useful.
- LWN’s kernel page has lots of great articles explaining kernel internals
- TLDP’s TLK. Somewhat outdated, but useful.
- Linux Journal’s KernelKorner has some excellent articles, most of which are freely available online.
- The Linux Kernel Hackers’ Guide, compiled by Michael K. Johnson of Red Hat fame. Includes among other
documents selected Q/A’s from the linux-kernel mailing list.
- The LinuxKernel HOWTO by Brian Ward.
- A completely new Kernelhacking-HOWTO at http://www.kernelhacking.org/.
- Various Kernel HOWTOs on specific questions, such as the BogoMips mini-HOWTO by Wim van Dorst.
- Various Linux HOWTOs at TLDP
Lists of links
- There’s a huge index of links at http://jungla.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html that has links to many resources.
- Another list of links maintained by Chris Gould. Mostly unsorted
Virtualization is a hot topic and the field is growing at a rapid pace. There are hundreds of papers on each sub-topic, and this list is intended to be a good starting point for someone starting in virtualization research.
These are good starting points, if you are just learning about virtualization
- Survey of virtualization machine research. Robert P. Goldberg. IEEE Computer, June 1974, pp 34-45. [PDF]. One of the oldest surveys about virtualization research.
- A Survey on Virtualization Techniques. Susanta Nanda and Tzi-cker Chiueh [PDF]. A more modern survey of virtualization techniques. The paper lists many of the techniques, but doesn’t really explain how they are different etc. Yet, this is a good read for budding virtualization researchers.
- Virtual Machines: Versatile Platforms for Systems and Processes . Jim Smith Ravi Nair. Amazon book link
These are the papers that will help in understanding fundamentals and concepts of virtualization.
- Xen and the art of virtualization. Barham et.al. This is the classic SOSP paper on para-virtualization. [PDF].
- When Virtual is Better Than Real. Peter Chen and Brian Noble. This is a great short article explaining the benefits of virtualization. ACM link.
- Disco: Running Commodity Operating Systems on Scalable Multiprocessors. E. Bugnion, S. Devine, and M. Rosenblum. This paper is considered to be the first paper that revived the virtualization concepts pioneered by IBM. ACM link.
- Running multiple operating systems concurrently on an IA32 PC using virtualization techniques. Kevin Lawton. This is a great article on the difficulties involved in virtualizing the x86 platform. link.
- Formal requirements for virtualizable third generation architectures. Popek and Goldberg. The classic paper explaining the requirements for virtualizing a specific ISA. ACM link. If you want light reading, check the Wikipedia explanation of the requirements.
- Scale and Performance in the Denali Isolation Kernel. Andrew Whitaker, Marianne Shaw, and Steven D. Gribble. Denali is another example of early virtualization papers that rejuvenated the research. Denali shows the containers (or vservers) concept, which is used in OpenVZ, KVM and VServers.
- The Exokernel Operating System Architecture. Dawson Engler’s thesis. [PS]. This is more of OS research, but a great read for understanding some of the techniques (thin hypervisor, pass through etc.) that are used in modern virtualized systems.
Virtualization of CPU is provided by a CPU scheduler that provides the illusion of multiple CPUs (or VCPUs). Scheduling has a long and rich history. Below are a few links in relation to virtualization.
- Xen’s credit scheduler. Xen’s credit scheduler is a proportional fair scheduler, which is pretty big improvement (especially in SMP environments) over the old SEDF scheduler.
- Xen’s scheduler is partly inspired by Carl Waldspurger’s lottery scheduling and the paper: Lottery Scheduling: Flexible Proportional-Share Resource Management by Carl Waldspurger is a must read. [PDF].
- Comparison of the Three CPU Schedulers in Xen – Ludmila Cherkasova, Diwaker Gupta, Amin Vahdat – Perfomance Evaluation Review. Vol 35, Number 2. Sept 3, 2007. This provides a great comparison of CPU schedulers and provides insight on how handling I/O is complicated. [PDF].
Network virtualization is gaining momentum in recent years. I would just point you to the following survey as a starting point.
- Survey of network virtualization. N.M. Mosharaf Kabir Chowdhury and Raouf Boutaba. [PDF]. This is pretty comprehensive and well-written survey on network virtualization. Mosharaf also has a nice network virtualization, listing lots of useful papers.
- Xen’s device driver virtualization.
- Virtualizing I/O Devices on VMware Workstation’s Hosted Virtual Machine Monitor. Jeremy Sugerman, Ganesh Venkitachalam, and Beng-Hong Lim. As the name suggests, this paper explains VMware Workstation’s mechanisms for virtualizing devices. [PDF].
- GPU Virtualization on VMware’s Hosted I/O Architecture. Micah Dowty, Jeremy Sugerman. This paper by VMware folks explains how VMware implements GPU virtualization. Great starting point for GPU virtualization. [PDF]. Andres’ paper on VMM indepedent graphics acceleration is a good follow-up reading.
- Memory Resource Management in VMware ESX Server. Carl A. Waldspurger. [PDF]. Classic paper on memory virtualization. This paper introduces memory ballooning, which is a great technique used in commercial platforms as well.
- Difference engine: harnessing memory redundancy in virtual machines. Diwakar Gupta et.al. [PDF]. A great follow up paper that talks about sharing memory across VMs. Incidentally, both these papers have received best paper awards.
- Live Migration of Virtual Machines. Clark et.al. One of the first academic papers discussing how to do live migration, which is implemented in Xen. VMware has supposedly implemented vmotion before this was published. [PDF]
- The design and implementation of Zap: a system for migrating computing environments. Osman et.al. Zap predates Xen paper and talks about sandboxing processes that can be migrated. This is more like container virtualization rather than full system virtualization. ACM link
- SnowFlock: rapid virtual machine cloning for cloud computing. Andres Lagar-Cavilla et.al. Greatp paper to read to understanding cloning of virtual machines. Snowflock link.
- Carl Waldspurger’s PhD thesis is a great place to start for understanding resource management. MIT link.
- Automated control of multiple virtualized resources. Padala et.al. Disclaimer: this is my own paper It’s a good place to start if you want to learn about automating resource management for virtual machines. The control theory aspects are a bit complex, but you can ignore them and focus on the issues in automating resource management. [PDF].
- Black-box and Gray-box Strategies for Virtual Machine Migration. Clark et.al. This is a follow up for live migration paper. The paper talks about strategies for automating the migration of VMs to meet specific goals. USENIX link.