Of Languages and Wheels

I want to take a brief detour in my journey in the land of software rants. This rant would have gone deeper into the notions of reflection and introspection and how they affect the ways in which systems can be extended, topics I raised last time in talking about the relations between data and metadata. I'll get to that, trust me, it's one of my favorite parts of computer science, but for now I want talk about the Tower of Babel and the Great Wheel of Reincarnation. These are old ideas in the computer world, but I'm beginning to think that a lot of practitioners haven't heard about them, or don't understand that they're getting caught by them.

The Tower of Babel arises wherever implementors have to use multiple programming languages and/or multiple programming paradigms in a single application. Now, that's not necessarily a bad thing; sometimes it takes more than the functionality you can find in just one tool to do the job. But to see how this can be a problem, look at the goulash that webapplications are often made of. For an example, I'm working on an application now, a manufacturing system that serves up assembly instructions, keeps track of the work flow at each station, and provides instructional content creation functions. It's not very complex now, though in later phases of development more functions will be added. In order to serve up dynamically created pages of content on a web page, in even this simple an application the following progranming languages were used:

• Java was the programming language for the model objects that represent the basic functionality of the application: assembly procedures, the steps of a procedure, the parts and the tools used in the assembly. It's also used for the business logic managers that manipulate the model, and the http request actions that control the generation of the pages*.
• An Object-Relation Mapping system called Hibernate is used to persist the model objects in the tables of a relational database; Hibernate is a Java class library that maps Java method calls to SQL Database queries. It is configured to define the persistent objects and the relational tables using a mixture of Java annotations on the model class source files and xml configuration files.
• The mapping of input HTTP requests to page display and navigation are controlled by a combination of different annotations and yet more xml files, part of another framework called Webwork.
• What is displayed on a page is programmed using Java Server Pages, a framework that allows to combine html, tag markup in several different tag dialects, fragments of Java code, and an expression evaluation language called OGNL.
• After the page display HTML is generated it is modified by Freemarker expressions and Sitemesh decorators which each affect some set of pages with common transformations.
  
• And some of the interaction on the pages is programmed in the html and tags of the JSP, while some is programmed to run on the client webbrowser using JavaScript and an interaction paradigm called AJAX.
• The whole application is integrated using a Java framework called Spring that is responsible for creating and initializing the objects used by all the other pieces, which again requires annotations and xml files for configuration.
  

To implement this application requires half a dozen different technologies and languages; it was originally built by 2 people, and is currently maintained and enhanced by 1 person, who has to use all those tools and be constantly watching for the problems that arise because of incompatibilities among them (and, believe me, they arise).

I haven't even mentioned the apache webserver and tomcat application server that have to be configured for the application, and if I were to describe, even at the highest possible level, what each of these pieces I have mentioned does, I'd be writing a book, not a blog entry.

Just as in the story of the Tower of Babel, the challenge is for all the parts to be able to communicate well enough that the whole can function, and the problem for a software developer is the same as for the general contractor on the Tower: get everybody to work together so that each worker's output is what the next worker needs as input, even when none of them understand each other directly.

The Tower of Babel, despite ita name, is actually more horizontal than vertical, if you think of layers of software in the usual manner as divided from each other vertically. There are several layers in the system I described above, but the various languages and technologies aren't necessarily confined to single layers. The second problem I want to talk about is more a vertical one, involving multiple, often many, layers.

The Wheel of Reincarnation sounds like something from Buddhist theology, but in this case it refers to an insight by Ivan Sutherland into what designers often do wrong in trying to optimize layered systems.

For this rant, the key point of Sutherland's discovery is this:

The pit fall of reincarnation of function is still with us. Sometimes the layers are inhomogeneous which leads to conceptual costs, namely it is too hard to understand the layered systems. Sometimes the inhomogeneities linger in legacy architectures long after their original justifications are forgotten.

In other words, be careful how you layer things, because the layers may become permanently buried under yet more layers. And sometimes the functionality of a layer or layers has to be undone or redone by layers above, because it doesn't quite match the requirements of the upper layer.

In fact, the Wheel of Reincarnation is the explicit policy of some computer technology developers. Intel, for instance, calls it the "Silicon Treadmill": first you write software to perform a task on the current generation of processors, then you identify key parts of the code which can be optimized by putting specialized functionality (asy, by writing microcode) into the processor, then you design a special chip to implement the functionality completely in silicon. An example is the logical layer of the ethernet protocol. At first, it was implemented completely in software, then general purpose processors were modified with extra on-chip memory for buffers, and finally a specialized chip was designed to do the entire logic layer functionality.

Now take the Tower of Babel and build layers on the Wheel using it. Do that iteratively for a few turns of the Wheel, and try to figure out what you've got. Finding out will involve using techniques very like the ones that archaeologists use to investigate the various cultures that have used a midden over time: carefully peeling away a layer at a time and examining how all the parts you find fit together. But even if you figure out how it works, you can't change it and make it work differently, or fix a bug in it. And that's why knowing about the Tower and the Wheel might make a designer think twice about decisions that could cause someone great distress in years to come.

---

* If you're familiar with the Model-View-Controller architecture for interactive applications; the model objects are the Model aspect and the actions are the Controller aspect (along with some JavaScript on the pages). The View aspect is comprised of the Java Server Pages, containing JSP tags, Webwork tags, OGNL expressions, Freemarker markup, and Sitemesh decorators.