Temporary technology limits shouldn’t become design patterns

Things are the way they are because they got that way

– Ken Boulding

This is a quote attributed to Boulding by Jerry Weinberg in his excellent Secrets of Consulting. My students will tell you that I am fond of bringing it into many a class discussion. It’s somehow more useful to me than trotting out Santayana’s observation about learning history.

We’re all hot for the new, new thing and innovation is always better when it’s disruptive. For all that, when thinking about new technology we would do well to spend time and thought understanding how we got to the current situation. Predicaments today grew out of decisions yesterday that made sense when they were made. If we want to leave fewer predicaments for our successors, we need to understand how those earlier logic trains derailed.

For example, consider the Y2K issue of the late 1990s; the transition from 1999 to 2000 raised the threat that computer programs might fail because of the common practice of storing only the last two digits of the year in computer files and databases. That common practice grew out of technology limits with storage and data access. There was also an unexamined assumption about the likely longevity of the systems being built in the 1960s, 70s, and 80s. Why worry about a problem that was decades in the future; surely the programs would be replaced long before then.

There seems to be a deeper issue at work that is troubling me. Design is always driven by constraints and requirements; what limits exist on what you hope to do. We fail to appreciate how technology constraints evolve very differently from other constraints. We treat certain limits on our designs as if they were integral to solving the problem at hand when they are actually temporary speed bumps in the technology. We solve the immediate problem—the dates in our files fit the space available—at the expense of creating a bigger problem later.

We talk of “technical debt” but I don’t think the notion works with less technical decision makers. Debt suggests you trade future interest payments for a solution now. What we do instead is lock the organization’s future within boundaries that disappear and make us look stupid to those who designed smarter.

Organizational decision making is often hampered by shortchanging the future. We don’t seem to have good methods for avoiding these mistakes in technology. Temporary technology limits become design constraints which become design assumptions that get baked into design patterns and then outlive their usefulness.

Project management, rocket science, and donuts

time to make the donutsDunkin Donuts ran a legendary ad campaign in the 1980s—“Time to Make the Donuts.” You can still find the ads online. They celebrated commitment to doing the work. What they ignored was the need to manage the work. In fact, they reinforced the idea that work and management were two distinct things.

For making donuts or widgets or Toyotas this is possibly a useful distinction. For the work that most of us do today, it does more harm than good. In the donut world work is about following recipes and management is making sure that workers follow the recipes. There’s a lot that can go into following and managing recipes; how accurately do workers follow the recipes, how fast, how many per shift.

But where do the recipes come from?

If you’d prefer to remain in a worker/manager universe, they arrive by innovation magic. Some mysterious, creative process serves up new recipes and processes to be plugged into the existing system. Perhaps there will be some change management pain and disruption to be absorbed. Perhaps you will turn to some specialist or consultant to carry out this odd work. Then, everyone can get back to work.

I think this explains something I always struggled to understand when pitching consulting projects or proposing change efforts within organizations. Decision makers always seemed to object to the project management line item and tasks in proposals and plans. For that matter, project team members weren’t keen on the process of managing the work; they wanted to focus on what they saw as the fun part of the work.

It all makes sense if doing and managing are two separate activities. But you can’t do that when they are intertwined. When the doing shapes what needs to be managed and the managing calls for picking your way through the doing then you are much more tightly coupled than the mythology of workers marching forward as managers point to the goal in the distance.

I’ve been fascinated by NASA’s recent flyby of Ultima Thule by the New Horizons Mission. If you’re interested, I’d recommend a recent Nova broadcast, Pluto and Beyond. One thing that struck me was that project management isn’t rocket science but successful rocket science certainly depends on effective project management.

This all matters because the work we are all doing these days is a lot closer to rocket science than it is to making donuts. We’d better start acting as if we believed that.

You can’t separate learning and doing

If I’m not careful when I introduce myself, “Jim McGee” gets heard as “Jimmy.” Curious as that was what I was called growing up. Outside of family and a few cousins, there were only two people who regularly called me “Jimmy” One was the director of the college theater group I was in. The other is a friend and colleague I worked with during my doctoral studies.

Although it’s a pretty common name, I take a certain perverse pride in laying claim to “jimmcgee” or “jmcgee” as a user name in places like gmail and twitter. It’s a marker of being an early adopter on multiple technology platforms.

You learn stuff by playing with it. But learning and play are suspect activities in most organizations. Outside of schools, the presumption is that you’ve learned what you need to know on someone else’s dime. Even in schools, you pay for the privilege of not knowing and being a learner.

This model works in a stable or slowly-evolving environment. There are places where you learn and places where you do. If you are in a doing place and the learning places are lagging, you might find it a good idea to create a private learning place to bridge the gap. But the idea of doing and learning remain separate.

I could argue that this is fundamentally wrong for all organizations and all times; that the separation of learning and doing is an artificial distinction that only works with the right confluence of factors and only for limited periods. We’re no longer in one of those periods.

We’ve been living through an extended period of accelerating change; it’s become an empty cliche. This is a cliche that you ignore at your peril. The half-life of what we know continues to shrink.

Half-life is a notion borrowed from nuclear physics. Radioactive elements and isotopes transform at a predictable rate; the transformation of Carbon-14 into Carbon-12, for example, is one of the facts that tell us that the world is more than 6,000 years old. The time it takes for half of the Carbon-14 in a sample to decay into Carbon-12 is the half-life and is a fixed and measurable rate.

Shifting back to knowledge and knowledge work, much of what we knew from our school days has decayed in similar fashion. By some estimates an engineering degree has a half-life of less than 10 years.

If you maintain the fiction that learning is something that occurs in learning places and is separate from doing, then you hire young computer scientists, move a handful into management, and replace the rest on a regular basis. A stupid management strategy, even it is appears to be a common one.

In work with a university research lab that was dealing with growing pains, I found the phrase “smart people doing smarter work” a helpful entry point toward a more effective response. There’s been a trend in knowledge intensive organizations toward hiring more people with Ph.D.s. At first glance, this can be viewed simply as seeking out people with more recent expert knowledge. The deeper truth is that a Ph.D. is someone who lives at the boundary of learning and doing; someone who understands that it is not, in fact, a boundary.

When I was a student, teachers were the people who had answers. If you had questions, you found the expert who had answers. When I was a consultant, I was an expert. When I reached the edge of what I knew, I looked for the next expert. Eventually, I reached a point where I ran out experts who knew. Since I was still operating from a learning and doing are separate things perspective, I went back to a learning place.

What I discovered was a community of fellow explorers who introduced me to a new practice, which was to say “I don’t know, let’s find out.” I was at the place with the half-life of knowledge problem was being created and attacked in parallel.

It’s certainly possible to treat this spot as just another place for experts. You can choose to be an expert at the edge, asking questions and passing answers back the chain to others who desire answers. What’s more interesting is to ask how to respond in a world where all of us operate closer to the edge of “I don’t know, let’s find out.” What if we looked to those already operating at the boundary between learning and doing as guides for traveling in this strange territory? What tools, practices, and habits of mind can we adopt to travel more effectively and safely in an environment where change is a feature not a bug?

Process and Ritual

ring of fireWhile there’s no obligation to explain your process to anyone, working it out for yourself does matter. There’s an observation from Aldo Leopold that’s pertinent:

The last word in ignorance is the man who says of an animal or plant, “What good is it?” If the land mechanism as a whole is good, then every part is good, whether we understand it or not. If the biota, in the course of aeons, has built something we like but do not understand, then who but a fool would discard seemingly useless parts? To keep every cog and wheel is the first precaution of intelligent tinkering.

In my previous post, I spoke about the broad spine of my process. There are aspects that feel important although I’ve yet to fully understand their fit and there are other elements whose purpose escapes me but I’m reluctant to set them aside. Learning, for example, appears in multiple non-obvious ways. I allocate substantial amounts of time to reading, of course; the world moves too quickly to rely simply on the accumulation of experience. I also try to have some topic I’m learning that is new to me; as a teacher, I want to always know what it feels like not to know something.

My writing practices have evolved over the years. I’ve gradually become more comfortable with letting writing evolve. When I wrote my first book, Ernst & Young provided us with an editor to work with us as we developed the manuscript. One day, John met me in my office. As I handed him the draft of my most recent chapter, I had to take a call from a client. As I spoke with my client, I was puzzled as John flipped past the first three pages and began reading the draft at the top of page four. When the call finished, I naturally asked John why. He gently explained that he had learned that I had a habit of clearing my throat for several pages and burying the lede; page four turned out to be a fairly predictable first place to look. I’ve gotten better at discovering my lede without outside assistance and putting it where I think it belongs intentionally.

Some find writing by hand a useful element of their process; my handwriting is both too slow and too illegible to help in that regard. What I have learned, however, is that it is valuable to capture snippets of ideas and phrasing as they occur to me. Technology makes that a more reliable process. What warrants further improvement is moving from snippet to finished product.

One practice that has helped at the outset of new projects is to write a “memo to self” that outlines a storyline of the effort as a whole. This is something other than a project plan. A project plan focuses on the sequence of tasks; the storyline is an attempt to find the intellectual thread that will connect facts, insights, and conclusions into a path forward.

I can’t necessarily explain why this works. But I treat it as a form of ritual. Whether you understand the ritual doesn’t matter; what’s important is that you commit to the practice. The open question is how to make these elements more visible to the people I am working with.

Quest for the organizing thought

crystallizationI thought of starting this piece with an observation that talking about my process risked spiraling out of control until I realized that feeling was, in fact, a part of my process. The most satisfying part of my work is bringing new things into existence. An essential step is generating enough raw material to ensure that something good and pleasing is likely to emerge

The process is about designing new capabilities. The domain is technology use to support organizational performance. The process and the domain combine to define a practice but it’s helpful to treat them separately.

My process has evolved over the years based on my exposure to other thinkers and on the lessons learned over multiple iterations of the cycle. In today’s vernacular, I would call it a process of design thinking. It starts with someone declaring that a problem exists. What follows is a classic problem-solving process;

  • searching out the facts to tease out a picture of “ground truth,”
  • immersion in the stew of ground truth and the broader context, adding new morsels and tidbits until there is a super-saturated solution,
  • flashing on a crystallizing phrase or formulation that causes insight to precipitate out of the super-saturated solution
  • elaborating the implications of the crystallizing formulation for what the next world needs to look like
  • bringing the next world into being in thought and deed

The process works in multiple environments. It has to be coupled with domain expertise and local environmental insight to be practical

There are two elements of this process that have proven to be important for me, although I haven’t seen them talked about much. This could be a hint that there’s something to be developed further, or it may simply reflect my idiosyncratic perspective. The first has to do with the step I’ve described as “immersion.” I think of it as a deliberate practice of staying in the question rather than pushing on quickly to old answers we find comfortable.

The danger of staying in the question, of course, is that you never move on; something that others warn against as “analysis paralysis.” This leads to that second element. My signal to move on in the process is when I hit on a short phrase that encapsulates my take.

For example, I was working with the director of a university research lab who was wrestling with the problem of how to better manage a group of professors, post-docs, and research analysts that had grown rapidly. After an initial round of interviews, I was reviewing my raw interview notes to see what I might have learned. The phrase that popped into my head was that the Lab Director was asking how could we help “smart people do smarter work.” Nothing exotic and certainly nothing Pulitzer Prize worthy, yet it was a signal to me that I had found a thread I could now work with.

The Persistent Myth of Five Year Plans

five year plan posterI marvel that the myth of the 5-year plan persists. Without the invention of the spreadsheet it might have already passed away. You would think that the “success” of 5-year plans in the former Soviet Union would have been a better clue. Regardless, managers continue to stress over their ability to predict the future and manage to those predictions.

There are only two ways to make plans that can survive a 5-year test. One is to operate in a stable/stagnant enough environment that the future can be seen in today’s reality. The second is to take so few risks that you convert your local environment into something that can pass for stable.

Smart organizations and smart managers approach planning differently. When we started Diamond in 1994 we talked about our 23 and 1/2 year plan. This “plan” was a simple picture that showed 3 1/2 years of high growth followed by a 20-year line of sustained, steady, growth. The point of that simple picture was to set a shared direction. The first task was to establish an organization and a culture. The second was to manage that organization for the long term.

There was no way and no point to make or believe predictions about what we would be doing in 5 years. But our aspirations gave us insights into the organizational capabilities we needed to build. Without the sense of direction, we would have no way to make choices about what to work on and what to ignore.

We were indeed trying to build a business but there were also insights to take over into building a body of work perspective. Chief among them was to focus on the skills and capabilities we needed to develop. Planning was about understanding the skills we had and the skills we hoped to develop next. If you have choices about what projects to take on, then one filter is what new paths does each project open up.

Dangerous magic–casting technology spells without understanding

I just gobbled up Charles Stross’s The Labyrinth Index, which is the 9th book in his Laundry Files series. The series is the hybrid that might result if you threw Ian Fleming’s, Robert Heinlein’s, and H.P. Lovecraft’s fiction into a blender. I love it; your mileage may vary.

The underlying premise of the series is that computation and applied mathematics is magic. Not magic in a metaphorical sense; real magic that can unleash demons and eat souls. The Laundry is secret British organization that tries to keep the average computer science student or random hacker from accidentally destroying the universe.

In my last post I talked about understanding the magic in technology. I remain puzzled by people who are content to not understand, who are content to accept something as mysterious and get on with it. As we continue to build and deploy technology we are putting magic in the hands of many who can inadvertently wreak havoc.

More often than not, they inflict harm only on themselves. But that is small comfort. Consider the example of the Morris Worm that crippled the early Internet because of a programming error that turned an unwise experiment into a catastrophe.

Clarke’s Third Law posits that “any sufficiently advanced technology is indistinguishable from magic.” Gregory Benford, another science fiction author suggested the variation, “any technology distinguishable from magic is insufficiently advanced.”

Those who market technology are fond of promising magic. Too many individuals and organizations treat that claim as operational guidance rather than puffery. Stross offers a reminder that we do so at significant risk. The answer lies not in further insulating people from the technology they employ, but in working to reduce the magic factor. We have to get better at making the technology beneath the magic accessible and understandable.

Building tiny bridges

Rope BridgeThe short version of the story I tell about why I decided to leave work to get a Ph.D. is that I needed to figure out why those stupid users weren’t taking advantage of the clever systems I was designing. That turns out to be a midpoint in a longer thread of trying to bridge two worlds that should be one.

The tagline for my blog is a quote from Dorothy Parker: “the cure for boredom is curiosity, there is no cure for curiosity.” In business, curiosity is acceptable only up to a point. You can be curious about the specific things you need to learn to tackle your next well-defined goal; free-floating curiosity is suspect.

Today, we are likely to label free-floating curiosity as attention deficit disorder and bring it under control. ADD is a problem of poor executive control, which implies that curiosity must be subordinate to focus.

I was fortunate to grow up in a time before we had ADD as an explanation for certain behaviors. I also had enough natural talent for school-like activities that my curiosity was nurtured and encouraged rather than constrained.

Sometime around the second grade I had my first collision between instinct and the conformity that most schools and teachers preferred. It was a Catholic school and I posed a problem for the nuns; I was clever but, as an eldest child, generally well-behaved. One morning, my teacher—a nun whose name is long lost to me—discovered that the workbook I was supposed to be maintaining for the last several weeks was empty. Who knows what I had been doing instead.

I was dispatched to the principal’s classroom with the empty workbook as evidence of my failure. The principal taught 8th grade students so my arrival provoked their amusement and my humiliation as intended. Knuckles rapped with the principal’s yardstick, I returned to my classroom. I was sent home at the end of the day with the empty workbook so that my mother would also know of my dissolute ways.

I don’t recall Mom being terribly upset. What I do recall is that I did the exercises I had neglected and completed the rest of the workbook that night as well. I returned the finished workbook the next morning.

Here’s where I got lucky. My teacher then and those who followed encouraged me to discover what else I could accomplish rather than force me to conform. None of them imposed their agenda or expectations. Once I got the work at hand done, I was free to explore whatever caught my attention. Their gift was to help me discover that learning was about questions not answers and, then, how to ask questions safely rather than ask safe questions.

In an alternate timeline, I might have gone strictly down the math and science path. I was certainly curious about things that were logical—puzzles that had answers, things that made sense. But, not only was I a Catholic in those days, I was catholic in my interests. I was equally interested in things that didn’t make sense to me. It turns out, of course, that many things don’t make sense in the way that mathematics and science make sense.

In my early years, I wasn’t well equipped to distinguish between what didn’t make sense because of my ignorance, what didn’t make sense because of its inherent complexity, and what didn’t make sense because of my limitations rather than my knowledge. I didn’t even have those buckets to work with.

Among the things that made less sense to me were people, especially in groups and organizations. I understood classrooms, I didn’t understand recess.

My intuitions about the logical, systemic, and structured world developed and improved pretty robustly. My intuitions about the human world were weaker and slower to develop.

One choice would have been to stay within the bounds of the logical. But my innate curiosity kept pushing me to make sense of all of my environment.

There’s a long-running split between the human and the systemic. C.P. Snow called it “The Two Cultures” back in 1959. Working that gap has been the place that suits me, although I prefer to work it at a more detailed level than Snow. Recognizing a big problem doesn’t yield a big solution. It calls for building bridges one piece at a time.

Learning and knowledge management in start up mode

 Becoming a founder of Diamond Technology Partners in 1994 was a textbook example of leveraging your network. In late 1993 I was ready for something new. I had published my first book and sent a copy to a former boss and mentor with a note that I would like to use his name as a reference as I started to look for that next new thing. Our phone rang early on a Saturday morning in October—this was when phones were still attached to walls. My wife answered and handed the phone to me.

It was Mel. He had read the book the night before after traveling all week. He quickly came to the point. We needed to meet as soon as we could synchronize our travel schedules. I could not use his name as a reference.

Within the week we were seated in a conference room at the Admiral’s Club at LaGuardia Airport. Mel shared one of the very first copies of the Diamond business plan. The plan included a projected organization chart with a box labeled “Chief Knowledge Officer/Chief Learning Officer.” The only job Mel was prepared to help me land was to figure out how to make that box work.

Three months later I began commuting from Boston to Chicago. By June my wife and two young boys were in a new house on the North Shore of Chicago trying to make new friends while I tried to figure out what that box might mean.

Putting knowledge management and learning on our to do list demonstrated no special insight; these were issues that all professional services organizations were tackling. Professional services firms live off their accumulating knowledge and their ability to disseminate that knowledge across the organization. Knowledge management and learning are core requirements.

Drawing different maps

What was less clear at the time, to me at any rate, was that the lessons from established firms could not simply be transplanted. The issues and problems they faced were problems of maturity; their solutions addressed their problems. We had to work out how our problems were different before we could begin to design appropriate solutions.

I think of this as drawing different maps because it reminds me that we were in different territory. That wasn’t immediately obvious. We had all come from established firms and had limited experience in start up environments. It was tempting and comforting to view our challenges as simply matters of resources and scale.

The limited resources of being a start up turned out to be an advantage. We had combined knowledge management and learning in one box on the organization chart because we didn’t have two people to warrant two boxes.

We did not have a “Diamond” way of working. We were actively creating one out of the McKinsey/Accenture/Booz/Bain/Ernst&Young/IBM/LEK ways we drew on from our collective experience. Our clients worked with us because they had problems that didn’t fit any of those established ways.

There’s the business you plan and there’s the business the market tells you that you are in – you had better learn to listen to the market. Rather than match problems to the solutions we understood from experience, we let the features of problems shape our approaches. Our learning and knowledge problems were tightly coupled.

Our knowledge management problem was not about how to codify and scale our work so that it could be reliably distributed. It was to push on the boundaries of what we knew to make room for more new ideas. Our fundamental knowledge management problem was how to extract and organize insights from the innovation that was taking place. Our problem wasn’t so much knowledge management as it was knowledge creation.

We had to learn how to draw the maps we needed as we went further into the wilderness. Our problem was more like being part of a research team collectively trying to figure out what we were seeing, what it meant, and what to do about it.

Expanding the learning problem

With the market pushing the entire organization to create new ways to work, the scope of our learning needs expanded. Who had to learn, what they had to learn, and how they had to learn it all changed.

In established firms, the primary learning problem is to equip younger staff with the skills and expertise of their elders. For us, everyone had to deal with learning as both learner and teacher. Junior staff came wanting and expecting to learn; not all senior staff shared that mindset.

We also had to reduce the lag between knowledge creation where we learned something for the first time to sharing that learning with everyone else who could take advantage of it. In more stable environments, knowledge creation could be followed by investments in codification and confirmation. In our environment, we were happy if there were a handful of relevant examples from prior work that could help us shape what to do next.

Circumstance forced us not only into learning by doing but often learning while doing. There’s a rich base of knowledge backing the case for the power of learning by doing. We had the good fortune to be able to tap the insights of two of the leading thinkers in this area—Roger Schank and Alan Kay.

You can incorporate doing into almost any learning situation. Learning while doing turned out to be a bit trickier. Thinking in terms of how to better support reflective practice is one element. Building After Action Reviews into our projects was another.

We also deliberately blurred the lines between learning and practice. In our presentation skills class for new hires, rather than craft exercises we polled the field for research tasks and assigned those to students. If, for example, we needed to gather intelligence on a prospective client that became an assignment for a team in the course. Further, we brought in experienced staff from the field to evaluate and judge the quality of the research and the quality of the presentations.

The success of these experiments encouraged us to knock down more of the barriers between the field and learning/knowledge management. We brought experts back from the field to serve as instructors. We asked those same experts to ground the courses in their current client experiences.

What makes all of this matter beyond the challenges of one start up is that more of today’s organizations face similar problems of external environments that change too quickly to be captured in the knowledge management and learning practices of more mature organizations. We have all become start ups and our practices need to adapt to remain relevant.

McGee’s Musings turns 17

The experiment continues. The first post here came in 2001. This post is somewhere north of 1500. The number is inexact because portions of the earliest postings here aren’t easily tracked down.

The past year has seen a bit of a reboot here in terms of posting frequency., For those of you who’ve been following along, what would you like to see more of? What would you like to see less of?