New Additions to the Family

We've been threatening to get a dog again, and now we've adopted not one, but two dogs. We'd originally planned to get them last Monday, but couldn't get everything ready in time, and then we realized that on Tuesday, the roofers would show up and decided to wait. Good decision.

Tuesday reminded me of that scene in The 5,000 Fingers of Doctor T where they're touring the dungeons, and Terwilliger shows them the poor wretch who's been imprisoned in a giant bass drum while someone beats on it. The roofers wanted to get the entire job done in one day, so they threw a crew of 6 or 8 at it, and went at it like gangbusters (or maybe eardrum busters). The dogs would have gone nuts; we almost did even though we went out for a couple of hours.

So Wednesday we went to the shelter, and after wanting to take home just about every dog we saw (there were a lot of Border Collie and Australian Shepherd mixes; our last dog was a Border and we fell in love with the breed), we found a pair of older dogs who'd been kenneled together. When we took them out for a meet and greet, they were friendly with us, and got along well with each other, so we adopted them.

There was a strategy in this. We've always had large working breed dogs; the Border, at 45 pounds, was the smallest dog we'd ever had. But now we're neither of us up to giving a dog like that the exercise it needs, and there's nothing worse you can do for the physical and mental health of a dog than not get it enough exercise. This is also true of children; in fact, there are so many similarities that we got a lot of our child-rearing techniques from a book on dog-training written by a Hollywood dog wrangler. So we were looking for small dogs; these guys are both small and past the young adult stage, so their need for exercise is (at least theoretically, see below) lower now.

One of the dogs is a Lhasa Apso female named Jompa, about 9 yo, and the other's a Rat Terrier male, Spencer, who's 6 yo. They're settling in fairly well, though the terrier turns out to be a bolter who can get through our chain link fence in seconds. The good news is that it's not running away, just the urge to explore, because he will come back. The bad news is we live on a busy street, and he doesn't have good car sense, so he's now on a leash whenever we go out, and will be until he's obedience trained.

ETA: our bolter is starting to learn to come back when called. I lost him at the Arboretum that we've been taking our dogs to since we moved to Portland 30 years ago, while trying to hang onto both leashes and get a stool sample for the vet (I need to get a couple of extra arms). But I yelled for him, and he came back and let me grab his leash again.

Neither of these dogs was ever trained in any way as far as we can tell, though they're housebroken and they recognize their names. Worse, the terrier has scars where a harness was put on too tight and left on him all the time, and he flinches from loud voices and criticism. So we have some work to do to win the dogs over (though after only twofour days they're both willing to cuddle and be petted by us, which is a very good sign).

We've taken them to the vet, and had some good news and some bad news. Jompa has an ear infection, a sore mouth where 2 teeth were extracted, and is a little overweight. On the other hand, she's healthy in other respects. Spencer has developed kennel cough since we brought him home from the shelter, and seems to have an allergy to grass. But we've got medicines to deal with most of this, and they are fairly good about taking them (Jompa doesn't like getting stuff squirted in her ear, but she doesn't get angry and snappish, and she doesn't hold a grudge afterward. All in all we're really pleased with the dogs, and we think they'll fit in very nicely.

Birthday Season!

It's the peak of the season now; Eva's birthday (7/29) has come and, until my birthday (7/31) we'll be the same age, then I pull ahead again. Jeremy's birthday (6/1), Eva's brother Steve's birthday (6/27), and his daughter Rachel's birthday (6/17)are all in June, Alex's birthday was a few days ago (7/19), and his wife Melissa bats cleanup on 8/4. Party ON, dudes!

Of Languages and Wheels

I want to take a brief detour in my journey in the land of software rants. This rant would have gone deeper into the notions of reflection and introspection and how they affect the ways in which systems can be extended, topics I raised last time in talking about the relations between data and metadata. I'll get to that, trust me, it's one of my favorite parts of computer science, but for now I want talk about the Tower of Babel and the Great Wheel of Reincarnation. These are old ideas in the computer world, but I'm beginning to think that a lot of practitioners haven't heard about them, or don't understand that they're getting caught by them.

The Tower of Babel arises wherever implementors have to use multiple programming languages and/or multiple programming paradigms in a single application. Now, that's not necessarily a bad thing; sometimes it takes more than the functionality you can find in just one tool to do the job. But to see how this can be a problem, look at the goulash that webapplications are often made of. For an example, I'm working on an application now, a manufacturing system that serves up assembly instructions, keeps track of the work flow at each station, and provides instructional content creation functions. It's not very complex now, though in later phases of development more functions will be added. In order to serve up dynamically created pages of content on a web page, in even this simple an application the following progranming languages were used:

• Java was the programming language for the model objects that represent the basic functionality of the application: assembly procedures, the steps of a procedure, the parts and the tools used in the assembly. It's also used for the business logic managers that manipulate the model, and the http request actions that control the generation of the pages*.
• An Object-Relation Mapping system called Hibernate is used to persist the model objects in the tables of a relational database; Hibernate is a Java class library that maps Java method calls to SQL Database queries. It is configured to define the persistent objects and the relational tables using a mixture of Java annotations on the model class source files and xml configuration files.
• The mapping of input HTTP requests to page display and navigation are controlled by a combination of different annotations and yet more xml files, part of another framework called Webwork.
• What is displayed on a page is programmed using Java Server Pages, a framework that allows to combine html, tag markup in several different tag dialects, fragments of Java code, and an expression evaluation language called OGNL.
• After the page display HTML is generated it is modified by Freemarker expressions and Sitemesh decorators which each affect some set of pages with common transformations.
  
• And some of the interaction on the pages is programmed in the html and tags of the JSP, while some is programmed to run on the client webbrowser using JavaScript and an interaction paradigm called AJAX.
• The whole application is integrated using a Java framework called Spring that is responsible for creating and initializing the objects used by all the other pieces, which again requires annotations and xml files for configuration.
  

To implement this application requires half a dozen different technologies and languages; it was originally built by 2 people, and is currently maintained and enhanced by 1 person, who has to use all those tools and be constantly watching for the problems that arise because of incompatibilities among them (and, believe me, they arise).

I haven't even mentioned the apache webserver and tomcat application server that have to be configured for the application, and if I were to describe, even at the highest possible level, what each of these pieces I have mentioned does, I'd be writing a book, not a blog entry.

Just as in the story of the Tower of Babel, the challenge is for all the parts to be able to communicate well enough that the whole can function, and the problem for a software developer is the same as for the general contractor on the Tower: get everybody to work together so that each worker's output is what the next worker needs as input, even when none of them understand each other directly.

The Tower of Babel, despite ita name, is actually more horizontal than vertical, if you think of layers of software in the usual manner as divided from each other vertically. There are several layers in the system I described above, but the various languages and technologies aren't necessarily confined to single layers. The second problem I want to talk about is more a vertical one, involving multiple, often many, layers.

The Wheel of Reincarnation sounds like something from Buddhist theology, but in this case it refers to an insight by Ivan Sutherland into what designers often do wrong in trying to optimize layered systems.

For this rant, the key point of Sutherland's discovery is this:

The pit fall of reincarnation of function is still with us. Sometimes the layers are inhomogeneous which leads to conceptual costs, namely it is too hard to understand the layered systems. Sometimes the inhomogeneities linger in legacy architectures long after their original justifications are forgotten.

In other words, be careful how you layer things, because the layers may become permanently buried under yet more layers. And sometimes the functionality of a layer or layers has to be undone or redone by layers above, because it doesn't quite match the requirements of the upper layer.

In fact, the Wheel of Reincarnation is the explicit policy of some computer technology developers. Intel, for instance, calls it the "Silicon Treadmill": first you write software to perform a task on the current generation of processors, then you identify key parts of the code which can be optimized by putting specialized functionality (asy, by writing microcode) into the processor, then you design a special chip to implement the functionality completely in silicon. An example is the logical layer of the ethernet protocol. At first, it was implemented completely in software, then general purpose processors were modified with extra on-chip memory for buffers, and finally a specialized chip was designed to do the entire logic layer functionality.

Now take the Tower of Babel and build layers on the Wheel using it. Do that iteratively for a few turns of the Wheel, and try to figure out what you've got. Finding out will involve using techniques very like the ones that archaeologists use to investigate the various cultures that have used a midden over time: carefully peeling away a layer at a time and examining how all the parts you find fit together. But even if you figure out how it works, you can't change it and make it work differently, or fix a bug in it. And that's why knowing about the Tower and the Wheel might make a designer think twice about decisions that could cause someone great distress in years to come.

---

* If you're familiar with the Model-View-Controller architecture for interactive applications; the model objects are the Model aspect and the actions are the Controller aspect (along with some JavaScript on the pages). The View aspect is comprised of the Java Server Pages, containing JSP tags, Webwork tags, OGNL expressions, Freemarker markup, and Sitemesh decorators.

Viva La Resistance!

I just sent an email to Senator Ron Wyden, D. Oregon, thanking him for attempting to stop the FISA bill, whose corporate immunity provisions are almost surely going to allow Bush, Cheney, and the rest of the gang that's currently raping and pillaging our way of life in the name of their own power and profit, to get out of the White House with all the swag they can carry and no chance of any sort of accounting.

If your Senator was one of the 15 who voted against cloture, send him or her an email, or a phone call, and let them know you appreciate what they did. If your Senator was one of the 80 who voted for cloture, to stop Senator Dodd's filibuster, let them know how despicable an act it was.

Follow the link above to Firedoglake, where the vote, and its significance to rescuing democracy from kleptocracy in the US is discussed in detail.

Voting against Cloture and for the Constitution:

Biden (D-DE)
Boxer (D-CA)
Brown (D-OH)
Cantwell (D-WA)
Dodd (D-CT)
Durbin (D-IL)
Feingold (D-WI)
Harkin (D-IA)
Kerry (D-MA)
Lautenberg (D-NJ)
Leahy (D-VT)
Menendez (D-NJ)
Sanders (I-VT)
Schumer (D-NY)
Wyden (D-OR)

Not Voting:

Senator Kennedy had a good excuse, he's seriously ill, but the rest of them?

Byrd (D-WV)
Clinton (D-NY)
Kennedy (D-MA)
McCain (R-AZ)
Obama (D-IL)

More Reflections On Software

A few days ago abi sutherland blogged about how artists and software designers are not in control of the outcome of their work, with particular application to devices and contrivances that are intended to be extended and customized by end users. That pushed some buttons in my head, and I posted a comment on the subject, which I've extracted below. I have some more to say on the subject, which I thought only fair to say on my own blog, so more posts will follow at less than the usual irregular intervals, I hope.

Oh-oh, you hit one of my current hot topics. I feel a rant bubbling up; I must take on my Aspect and wield my Attribute.

The system I'm working on now has some major design problems because some fundamental issues about valid character sets and the use and communication of metadata mixed with data weren't addressed at all. These things are central to building a system that's flexible, extensible, and allows mashups to be created without knowledge of the system internals (preferably without any sort of deep technical knowledge at all*).

I've been thinking about the sort of design criteria that are important in software that can meet these requirements. Here are my first thoughts; I'll continue on my own blog as they develop, but you got the juices flowing now, so I'll begin here if you don't mind.

1. Quoting is vital. It must be easy to encapsulate any string of valid data (e.g., text characters, images, the basic vocabulary of a web page) so that the system can treat it as an opaque object except where the data really is meaningful. It shouldn't be necessary for an admin or someone extending the system to deal with escaping characters significant to parts of the system, like '&' and ';' in HTML, or ''' or '"' in SQL literal strings, or (oh, help me Noshabkeming!) spaces in filenames. Always remember that a filename can come from most anywhere.

2. Metadata is data to any system that doesn't have to process it. Opaque objects have to remain opaque unless the software really needs access to the insides. Accidental access is a good creator of truly diabolic bugs. I created a test case yesterday with an object whose name contained an apostrophe. This caused the web application that read the name from the database to crash.

3. Objects shouldn't have to have knowledge of each others' lifecycles. If an object is created in one system and another system creates another object to collaborate with it, then either the first has to exist for as long as the second needs it (and that's often not easy or even possible to guarantee), or the second system has to make sure that it copies everything the second object needs from the first, without creating any issues of staleness of the copies during the life of the second object (and that's as close to an intractable problem in a distributed system as I know about). Anybody who's ever slaved over request-scoped webapp variables has been bitten by this one.

Akk! This is growing from a screed into a tome; not what I intended. I'll stop now and go get ready for work. Let me know what you think of this.

* Librarians are technically-oriented, but don't necessarily have specific knowledge about the administration of software, especially web-based systems. The manufacturing workers who administrate my system are not technically sophisticated at all; they're bright people, but don't have technical educations, so they're not familiar with some basic concepts that would make it easy for them to create a mental model of what they can do with the software. And I can't expect to train them in something that's irrelevant to their jobs.

Blog Against Torture - Not Just Today

No matter how busy I may be, I'm not too busy to say something against the use of torture on the day that Bear reminds me is Blog Against Torture Day. Nor will I forget the subject after the day ends.

An example of why torture is not only evil but also profoundly counter-productive. This is something I wrote, a draft of a newspaper ad, after hearing about the case of Abdallah Higazy. One important thing to remember is that, come to the end of the case, the FBI was more concerned that their mistakes would be covered up than that the job of securing the country would be performed effectively.


The US Government coerced a false confession from an innocent man by threatening to have his family turned over to be tortured by Egyptian security police. Is this what you want your government doing in your name?

Abdallah Higazy was interrogated by the FBI on the basis of information that the Justice Department later admitted was partly incorrect, partly the lies of an informant who has been charged with lying to the FBI. In order to get a confession, Higazy was told his family would be arrested and that the interrogator would “make sure that Egyptian security gives [his] family hell.” Higazy was later released because the original information that led to his arrest was false.

Ask yourself what you would do to protect your own family from a security service with a world-wide reputation for torture and other barbaric practices, a security service that trained many of Saddam Hussein’s torturers. Would you give a false confession, knowing it would probably destroy your own life, if you were told it would keep your family from harm? Abdallah Higazy did. So the FBI obtained a false confession, that they had to know was suspect because of the way it was obtained, and they wasted resources on this case at a time when all of our resources need to be focused on finding and eliminating the terrorists who have caused the US so much harm.

But even more, is threatening a father with the rape and torture of his daughter an act worthy of the United States of America? We believe it is not, that it is an act worthy of Saddam Hussein, the very dictator the US committed military forces to remove from Iraq. Furthermore, we believe it is an act so vile that we want to make clear to the world that we do not condone it and will not allow our names and our honor as citizens to be used to justify it.

We, the undersigned, believe that what the US government does reflects on the honor and integrity of US citizens. As citizens we object to our government committing illegal and immoral acts, and we want other citizens to be aware of what their government is doing so that they too can object. We urge you to take a stand on this issue, to tell your Congressional Representatives and Senators that this kind of behavior is not what America was founded for.