Down the Rabbit Hole with Sketchbook and Camera

I started this post more than 2 years ago, and put it aside after I stalled out in the writing. Then a few weeks later I had back surgery with a couple of months of recovery during which I forgot all about this post. I've patched it up a bit, but it's far from finished. I'd like to get back to it some day, but I don't know when that might be, and in the meantime I'd like to see other people thinking about some of these issues. So here it is; make of it what you will.

--------

Every once in a while I discover an interesting question that I want to answer (or attempt to answer, at any rate; some of these questions have been hanging around for thousands of years, so it's unlikely that I'm going to figure out anything definitive about them), and in reading or otherwise researching the question I discover other interesting questions, and the subject ramifies into what would be a dissertation if I were in graduate school.  Not too long ago this happened again.  Reading a book that had been on my "to read" list for almost 15 years, I started thinking about some issues that I've been thinking about on and off for most of my professional career.

The book was "Abstracting Craft" by Malcolm McCollough (see the references after the fold for a link).  It argues for a notion of digital craft (such as for computer-based architectural design, or product design)  based on  manual skill similar to the skills of a potter or a draftsperson.  This notion tickles an old interest of mine in just what a tool is, and how we humans form such tight linkages with our tools that we seem to somehow include them directly into our thinking and acting.  It also brought up some ideas for an SF story I've been thinking of writing.

Read on to follow me down the rabbit hole.

First some history on how I got involved in thinking about these subjects.  I worked for most of the 1980s on computer graphics display and input systems for Intel and Tektronix.  From 1983 to 1985 and 1986 to 1989 I worked as the project leader for design and implementation of 2 dimensional and 3 dimensional graphic input systems for computer workstations.  I spent a lot of that time learning what Tektronix' customers were doing with the existing graphic products, and finding out what new things they wanted to be able to do.  And I learned a lot about how computer graphics input and display have to work together to give a user the sense of having input and output connected together, so that manipulating an input device like a 3D trackball or tablet can move a complicated 3D shape without lagging or skipping or otherwise breaking the illusion that the user's hand is connected to the computer's output.

Thinking about how that all worked led to me ask myself how a user came to think of a tool as an extension of her hand.  The concrete case I thought a lot about was how I drive a car.  When I first start to drive a particular car I drive more slowly and carefully than I would with a car I am already familiar with, because I haven't yet internalized the size and handling characteristics of the new car.  But once I'm familiar with it (and it only takes a few days of regular driving) I no longer have to think consciously about how wide the car is, or what its turning radius is.  Those characteristics have become internalized in some way that allows the conditioned reflexes and unconscious habits I've already learned in driving past cars to make use of them.  And that feels to me as if the car is an extension of my body; I ignore the fact that my foot presses on the brake to slow or stop the car and think of it instead as stopping as a direct result of my decision to stop.

Many of the mental tasks the human mind is so good at involve making models of physical objects: maps of the space around us created from visual input, models of the behavior of people we meet created from observations of them, and schemas of stereotypical situations in which we've found ourselves in the past.   We use these models to predict future reactions to our own actions, and to plan those actions.  So, I wondered, perhaps my mind builds models of the tools I use, and inserts those models into models of my own body, creating an extended model I use to predict how the combination of me plus a tool will behave when I use it.  That would mean there's a model in my mind somewhere of how my car's brakes behave when I press the brake pedal, and it's connected to a model of my foot when I drive the car.

McCollough carries the idea of digital tool well beyond the simple case of a graphic input device creating a visual artifact, such as a drawing or an architectural design;  whenever a system can modify an output fast enough for a user to get the feeling of continuity of effort that manual skill requires, he believes that real craft is possible.  This jibes with my own belief that many kinds of tools that manipulate symbolic media, such as programming tools, also allow a form of craft.  That expands the class of "tools" to include many kinds of virtual devices whose behavior can be modeled by a mind and composed with a user's internal model of herself.

References:

1. Abstracting Craft: The Practiced Digital Hand" by  Malcolm McCullough
2. "Simulation and Its Discontents" by Sherry Turkle