Posts tagged Research

My friend Pranav wrote a blog post on a topic that we talked long ago:Is Theory better than Systems? As he mentioned, we both were passionate and some what arrogant in arguing the merits of one over the other.

I have matured a lot in the last few years, and I can claim to have done some real “Systems research” finally. Interesting post, Pranav, but it misses some important points, may be because you haven’t experienced them yet. Here it goes.

Pranav says,

Definition: Discipline A is superior to discipline B if it improves the lives of a greater number of people.

Good definition. Though vague, it’s enough for our arguments. Let’s take your first point. CS Theory is much closer to Math, so it’s superior. There’s no doubt that it’s closer to Math, but how did it become superior ? It sure is easier to code than prove a theorem (may be), but systems research is not about coding or implementation.

Next, talking about first principles, you are absolutely right. If we were to build a system from first principles, it will certainly be more general and widely applicable. But, where are those first principles ? In my research, I (and my mentor Xiaoyun Zhu, a control theorist) argue that there are no first-principle based models in computer systems. If we had these models, we would simply throw those into an intelligent computer and it will spit out the system specifications for us. Coming back to superiority, Would you rather have years and years of useless results, that never get into practice or a system that works and may help in actually building those first principles ? If we are talking about improving the lives of people, a good working system actually brightens the lives of a lot more people including theory researchers. Let’s take a concrete example, sure NP =? P is a problem that is of extraordinary importance. Solving it would change the face of computer science. It surely is a lot more difficult problem than designing UNIX, in fact there’s no comparison. BUT, how did the theory researchers in the past 20 years help the lives of people ? Listing a few results like RSA algorithm doesn’t cut it. There are equally as significant results in Systems (UNIX, C, Ethernet, TCP/IP etc.)

In fact, systems research affects a lot more people than theory research. If we solve the problem of NP =? P, it surely will affect a lot more people, but in the process of proving that, theory researchers contribute very little to the lives of actual people. To give another example, it required systems people to tell all those theory people working for years on millions of scheduling algorithms that Round-robin works better than many others because of its simplicity. It would have been much simpler, if the first theorist who thought of a scheduling algorithm actually built a real system. We could have made better use of those “sharper” brains.

Pranav says,

Definition: Discipline A is superior to discipline B if it is harder to publish a result in leading conferences/journals in discipline A than it is to publish an equivalent result in discipline B.

I had to have a big laugh over this. There are a lot more systems conferences/journals than theory conferences/journals because of its nature. It’s certainly easy to publish any random idea in any random conference probably even in theory research. It’s very difficult to publish in the “best” systems conferences as it is very difficult to publish in the best theory conferences. In fact, since more people are working in systems, it’s a lot more difficult to publish in systems. It takes a lot more work to design, build and evaluate a system compared to writing off a random theory idea. Theorists don’t have to prove that it really works, and there will be very few people who actually know the nitty, gritty details of the proof some one worked on. On the other hand, if you write a systems paper, it will be torn to pieces by many researchers who have actually built similar systems. The devil is in the details.

Let me digress a little bit. Long ago, Pranav talked about how easy it is to read a systems paper for any one working in computer science where as a theory paper would not even be understandable to many. There lies the superiority of systems, I would say. It sure is easy for any one to look at the new system, use the system, even experiment with the system. This actually improves the system a lot considering that every computer scientist can understand what it’s doing. BUT, It really takes brilliant systems researchers to design and build it. Many computer science people (including a few systems researchers) don’t actually understand the real details of building a system that actually works. These are NOT implementation details, far from it.

To conclude, there seems to be some confusion over the superiority of results obtained in Systems and Theory. There’s absolutely no doubt that the results that Theory brings once in a while are extraordinary and very significant. BUT, they are few and come out rarely. As Pranav mentioned, I have utmost respect for any one who really believes that they can solve these fundamental problems. While theorists ponder over these grand problems, there are equally tough problems in systems. There is no corresponding theory for them, because there are no first principles. As a result, significant systems results also come at a very slow rate, but systems research contributes a lot more to the good of computer science and actually helps the lives of common people.

Philosophically speaking, both theorists and systems researchers are working to make computer science better, systems researchers contribute to smaller problems a lot more while doing this than theory researchers.