Review – Measure What Matters

Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs. John Doerr

I have a collection of T-shirts received as Christmas gifts from my wife. In that collection is one that expresses an all-too-common management practice, “the beatings will continue until morale improves.”

Most management books strive to offer better advice. Long-time venture capitalist, John Doerr, has been an evangelist for OKRs, an approach developed at Intel and in vogue across Silicon Valley. “Measure What Matters” is Doerr’s effort to package the approach for wider dissemination. It’s an approach well worth understanding.

OKRs is short for Objectives and Key Results. Doerr characterizes it as “a collaborative goal-setting protocol for companies, teams, and individuals.” Doerr and the other advocates for OKRs are engineers by training and temperament; they think in terms of elegant, interlocking systems. Well-designed OKRs are just that; the kicker is that “well-designed” is the hard part and it’s easy to miss that in the cheerleading.

Doerr defines the system as:

“A management methodology that helps to ensure that the company focuses efforts on the same important issues throughout the organization.” An OBJECTIVE, I explained, is simply WHAT is to be achieved, no more and no less. By definition, objectives are significant, concrete, action oriented, and (ideally) inspirational. When properly designed and deployed, they’re a vaccine against fuzzy thinking—and fuzzy execution. KEY RESULTS benchmark and monitor HOW we get to the objective. Effective KRs are specific and time-bound, aggressive yet realistic. Most of all, they are measurable and verifiable.

Who could object?

The rest of the book elaborates on this and walks us through a number of case studies of the challenges of putting the theory into practice. Doerr is quite explicit that the “regimen demands rigor, commitment, clear thinking, and intentional communication.”

He acknowledges that “ideas are easy—execution is everything.” Digging into the case studies and thinking about the richer stories that are compressed into the retelling is essential. Doerr tells us this if we are paying attention; “what’s neat about OKRs is that they formalize reflection.”

Today’s organizations and markets are too complex for a single mind to comprehend. You need to engage as many minds as possible to wrestle with the complexity. OKRs give you a process and a language system but success depends on the conversations that you have with the language.

Doerr only hints at the communications challenges that lurk underneath. How do you distinguish a “goal” from an “objective?” I’ve seen that conversation go in circles for hours. You can have theological debates about OKRs as well. It tempting to talk about the theology, but progress comes when you bring the conversation down to the level where you agree on specifics.

If you’re so inclined you may want to start with Doerr’s TED Talk but “Measure What Matters” needs to be on your reading list.

Process and Ritual

ring of fireWhile there’s no obligation to explain your process to anyone, working it out for yourself does matter. There’s an observation from Aldo Leopold that’s pertinent:

The last word in ignorance is the man who says of an animal or plant, “What good is it?” If the land mechanism as a whole is good, then every part is good, whether we understand it or not. If the biota, in the course of aeons, has built something we like but do not understand, then who but a fool would discard seemingly useless parts? To keep every cog and wheel is the first precaution of intelligent tinkering.

In my previous post, I spoke about the broad spine of my process. There are aspects that feel important although I’ve yet to fully understand their fit and there are other elements whose purpose escapes me but I’m reluctant to set them aside. Learning, for example, appears in multiple non-obvious ways. I allocate substantial amounts of time to reading, of course; the world moves too quickly to rely simply on the accumulation of experience. I also try to have some topic I’m learning that is new to me; as a teacher, I want to always know what it feels like not to know something.

My writing practices have evolved over the years. I’ve gradually become more comfortable with letting writing evolve. When I wrote my first book, Ernst & Young provided us with an editor to work with us as we developed the manuscript. One day, John met me in my office. As I handed him the draft of my most recent chapter, I had to take a call from a client. As I spoke with my client, I was puzzled as John flipped past the first three pages and began reading the draft at the top of page four. When the call finished, I naturally asked John why. He gently explained that he had learned that I had a habit of clearing my throat for several pages and burying the lede; page four turned out to be a fairly predictable first place to look. I’ve gotten better at discovering my lede without outside assistance and putting it where I think it belongs intentionally.

Some find writing by hand a useful element of their process; my handwriting is both too slow and too illegible to help in that regard. What I have learned, however, is that it is valuable to capture snippets of ideas and phrasing as they occur to me. Technology makes that a more reliable process. What warrants further improvement is moving from snippet to finished product.

One practice that has helped at the outset of new projects is to write a “memo to self” that outlines a storyline of the effort as a whole. This is something other than a project plan. A project plan focuses on the sequence of tasks; the storyline is an attempt to find the intellectual thread that will connect facts, insights, and conclusions into a path forward.

I can’t necessarily explain why this works. But I treat it as a form of ritual. Whether you understand the ritual doesn’t matter; what’s important is that you commit to the practice. The open question is how to make these elements more visible to the people I am working with.

Quest for the organizing thought

crystallizationI thought of starting this piece with an observation that talking about my process risked spiraling out of control until I realized that feeling was, in fact, a part of my process. The most satisfying part of my work is bringing new things into existence. An essential step is generating enough raw material to ensure that something good and pleasing is likely to emerge

The process is about designing new capabilities. The domain is technology use to support organizational performance. The process and the domain combine to define a practice but it’s helpful to treat them separately.

My process has evolved over the years based on my exposure to other thinkers and on the lessons learned over multiple iterations of the cycle. In today’s vernacular, I would call it a process of design thinking. It starts with someone declaring that a problem exists. What follows is a classic problem-solving process;

  • searching out the facts to tease out a picture of “ground truth,”
  • immersion in the stew of ground truth and the broader context, adding new morsels and tidbits until there is a super-saturated solution,
  • flashing on a crystallizing phrase or formulation that causes insight to precipitate out of the super-saturated solution
  • elaborating the implications of the crystallizing formulation for what the next world needs to look like
  • bringing the next world into being in thought and deed

The process works in multiple environments. It has to be coupled with domain expertise and local environmental insight to be practical

There are two elements of this process that have proven to be important for me, although I haven’t seen them talked about much. This could be a hint that there’s something to be developed further, or it may simply reflect my idiosyncratic perspective. The first has to do with the step I’ve described as “immersion.” I think of it as a deliberate practice of staying in the question rather than pushing on quickly to old answers we find comfortable.

The danger of staying in the question, of course, is that you never move on; something that others warn against as “analysis paralysis.” This leads to that second element. My signal to move on in the process is when I hit on a short phrase that encapsulates my take.

For example, I was working with the director of a university research lab who was wrestling with the problem of how to better manage a group of professors, post-docs, and research analysts that had grown rapidly. After an initial round of interviews, I was reviewing my raw interview notes to see what I might have learned. The phrase that popped into my head was that the Lab Director was asking how could we help “smart people do smarter work.” Nothing exotic and certainly nothing Pulitzer Prize worthy, yet it was a signal to me that I had found a thread I could now work with.

The Persistent Myth of Five Year Plans

five year plan posterI marvel that the myth of the 5-year plan persists. Without the invention of the spreadsheet it might have already passed away. You would think that the “success” of 5-year plans in the former Soviet Union would have been a better clue. Regardless, managers continue to stress over their ability to predict the future and manage to those predictions.

There are only two ways to make plans that can survive a 5-year test. One is to operate in a stable/stagnant enough environment that the future can be seen in today’s reality. The second is to take so few risks that you convert your local environment into something that can pass for stable.

Smart organizations and smart managers approach planning differently. When we started Diamond in 1994 we talked about our 23 and 1/2 year plan. This “plan” was a simple picture that showed 3 1/2 years of high growth followed by a 20-year line of sustained, steady, growth. The point of that simple picture was to set a shared direction. The first task was to establish an organization and a culture. The second was to manage that organization for the long term.

There was no way and no point to make or believe predictions about what we would be doing in 5 years. But our aspirations gave us insights into the organizational capabilities we needed to build. Without the sense of direction, we would have no way to make choices about what to work on and what to ignore.

We were indeed trying to build a business but there were also insights to take over into building a body of work perspective. Chief among them was to focus on the skills and capabilities we needed to develop. Planning was about understanding the skills we had and the skills we hoped to develop next. If you have choices about what projects to take on, then one filter is what new paths does each project open up.

Questions about questions

One of my professors, Warren McFarlan, made an observation about the choice of questions that has stuck with me. He argued that you could either ask questions that would get you published or you could ask questions that mattered.

He might have been a bit more diplomatic in his phrasing but that was the essence. Were you going to be driven by safety or by curiosity?

Most of education and society drives our questions to safety. We learn which questions lead to the answers that our teachers and managers are looking for. “Will this be on the test?” is the default question. The ways we suppress curiosity while pretending to encourage it are limitless.

I had a classmate in business school who liked to chat with our professors after class, which earned him a reputation as a brown noser. The notion that you might be genuinely curious about the subject was a foreign concept to most of our classmates. I got to know Phil as the year went on and discovered that he already had a Ph.D. in the History and Philosophy of Science. I may have been the only one in our section that he revealed this to; he believed, rightly I suspect, that this would only brand him further.

Curiosity is a dangerous value; it has killed more than cats. Suppose you also believe it is important regardless of the risks. How do you encourage, promote, and nourish real curiosity in environments that fear and suppress it?

How do you learn to ask questions that matter?

I’m making a claim here that I’ve avoided this problem and navigated the system with my curiosity intact. How did I manage that? More importantly, are there any lessons to learn?

School was a place where most things came pretty easy. But the culture still rewarded right answers and compliant behavior. As I listen to others tell tales of the pressures to achieve they faced from parents and teachers, they strike me as foreign. I was certainly praised when I did well, but I never felt pressured. My mom was always a believer. She was pleased with my results. But, she never talked about expectations. I imagine that with my six younger siblings to wrangle she was mostly relieved that I was operating on my own.

Looking back, my dad was pretty tricky. It was never “were you the best?” The question was “does this represent your best work?” If the answer was yes, the actual grade was immaterial; if the answer was no, then an A was no defense. That got me through high school and by the time I reached college I was comfortable chasing my interests instead of grades. I took courses based on my interests at the time, not based on my grade point average. As I often tell my students today, I managed to collect at least one of every possible grade.

Grades served as feedback, not measures of my value.

My work in the theater taught a second important lesson. Working on stage productions was always about scarce resources and hard deadlines. You can generally negotiate an extension on a class assignment; you can’t move opening night. On a show, the question is always “does this make what the audience sees better?” That leads to a second question, “if the audience can’t see it, why are you doing it?” There’s no good reason to paint the back side of a set.

The question is not about effort; it’s about value. We track effort because we can’t always wait to measure value; effort can be an appropriate early warning signal. Demanding maximum effort is a lazy manager’s way to avoid thinking about the value you are seeking to create.

The old adage is “if something is worth doing, it’s worth doing well.” I encountered an improved version on the bulletin board of a doctor friend:

If something is worth doing, it’s worth doing well enough for the purpose at hand. It is probably wrong, and certainly foolish, to do it any better.”

Throwing resources at problems is rarely an effective strategy despite the frequency with which it is employed. Acceptable as a strategy if you are resource rich, questionable if the problems and opportunities exceed the resources available. Getting the most leverage out of resources starts with learning how to ask better questions.

Dangerous magic–casting technology spells without understanding

I just gobbled up Charles Stross’s The Labyrinth Index, which is the 9th book in his Laundry Files series. The series is the hybrid that might result if you threw Ian Fleming’s, Robert Heinlein’s, and H.P. Lovecraft’s fiction into a blender. I love it; your mileage may vary.

The underlying premise of the series is that computation and applied mathematics is magic. Not magic in a metaphorical sense; real magic that can unleash demons and eat souls. The Laundry is secret British organization that tries to keep the average computer science student or random hacker from accidentally destroying the universe.

In my last post I talked about understanding the magic in technology. I remain puzzled by people who are content to not understand, who are content to accept something as mysterious and get on with it. As we continue to build and deploy technology we are putting magic in the hands of many who can inadvertently wreak havoc.

More often than not, they inflict harm only on themselves. But that is small comfort. Consider the example of the Morris Worm that crippled the early Internet because of a programming error that turned an unwise experiment into a catastrophe.

Clarke’s Third Law posits that “any sufficiently advanced technology is indistinguishable from magic.” Gregory Benford, another science fiction author suggested the variation, “any technology distinguishable from magic is insufficiently advanced.”

Those who market technology are fond of promising magic. Too many individuals and organizations treat that claim as operational guidance rather than puffery. Stross offers a reminder that we do so at significant risk. The answer lies not in further insulating people from the technology they employ, but in working to reduce the magic factor. We have to get better at making the technology beneath the magic accessible and understandable.

Understanding and explaining the magic

Skokie Benefit Rehearsal 2003The place where technology, organizations, and people come together has been a continuing focus of my work. That interest was birthed in stories of the wonders and dangers of fantastic new inventions. Like a lot of future scientists and engineers I was raised on the stories of Isaac Asimov, Arthur C . Clarke, and Robert Heinlein.

Somewhere in my early, unsupervised, reading I encountered Arthur C. Clarke’s “Profiles of the Future,” long before I was mature enough to grasp much of it. I’ve since returned to it multiple times over the years. One thing stuck from the earliest, Clarke’s Third Law:

Any sufficiently advanced technology is indistinguishable from magic

That notion gestated in the back of my mind as two strands played out. First, I encountered and learned about technology in multiple manifestations; mathematics, physics, carpentry, programming, electricity.

Second, I created magic in the form of live theater. Group effort focused on creating illusions that touched the heart observed from a spot where all the illusion was exposed. It’s an odd experience to watch a singer bring an audience to tears while listening for the key change that is the cue to dim the lights and adjust their color; all of which reinforces her voice  and triggers those tears night after night.

Where this has taken me is showing people that the magic is understandable. Revealing the technology that creates the magic gives people power to create their own magic and power lets you solve problems.

There is a moral question of whether you hoard or distribute power. I land on the distributing side; the problems we face need all the power we can collectively muster. Hoarding crimps the flow of total power we need.

You transform magic into technology in stages. The first is to shift perspective and take people behind the scenes. That involves telling the story twice; once from the magic side and again from the technology side. Next, you break down the experience into its component parts and reveal the seams. Finally, you help people learn to create and assemble their own designed experiences.

The venue for this transformation can take multiple forms. It can be as simple as a telling of the tale from the right perspective. Or, it can be a guided tour. With enough time and resources, the best choice can be a “build your own” experience with a veteran guide at your side.

Building tiny bridges

Rope BridgeThe short version of the story I tell about why I decided to leave work to get a Ph.D. is that I needed to figure out why those stupid users weren’t taking advantage of the clever systems I was designing. That turns out to be a midpoint in a longer thread of trying to bridge two worlds that should be one.

The tagline for my blog is a quote from Dorothy Parker: “the cure for boredom is curiosity, there is no cure for curiosity.” In business, curiosity is acceptable only up to a point. You can be curious about the specific things you need to learn to tackle your next well-defined goal; free-floating curiosity is suspect.

Today, we are likely to label free-floating curiosity as attention deficit disorder and bring it under control. ADD is a problem of poor executive control, which implies that curiosity must be subordinate to focus.

I was fortunate to grow up in a time before we had ADD as an explanation for certain behaviors. I also had enough natural talent for school-like activities that my curiosity was nurtured and encouraged rather than constrained.

Sometime around the second grade I had my first collision between instinct and the conformity that most schools and teachers preferred. It was a Catholic school and I posed a problem for the nuns; I was clever but, as an eldest child, generally well-behaved. One morning, my teacher—a nun whose name is long lost to me—discovered that the workbook I was supposed to be maintaining for the last several weeks was empty. Who knows what I had been doing instead.

I was dispatched to the principal’s classroom with the empty workbook as evidence of my failure. The principal taught 8th grade students so my arrival provoked their amusement and my humiliation as intended. Knuckles rapped with the principal’s yardstick, I returned to my classroom. I was sent home at the end of the day with the empty workbook so that my mother would also know of my dissolute ways.

I don’t recall Mom being terribly upset. What I do recall is that I did the exercises I had neglected and completed the rest of the workbook that night as well. I returned the finished workbook the next morning.

Here’s where I got lucky. My teacher then and those who followed encouraged me to discover what else I could accomplish rather than force me to conform. None of them imposed their agenda or expectations. Once I got the work at hand done, I was free to explore whatever caught my attention. Their gift was to help me discover that learning was about questions not answers and, then, how to ask questions safely rather than ask safe questions.

In an alternate timeline, I might have gone strictly down the math and science path. I was certainly curious about things that were logical—puzzles that had answers, things that made sense. But, not only was I a Catholic in those days, I was catholic in my interests. I was equally interested in things that didn’t make sense to me. It turns out, of course, that many things don’t make sense in the way that mathematics and science make sense.

In my early years, I wasn’t well equipped to distinguish between what didn’t make sense because of my ignorance, what didn’t make sense because of its inherent complexity, and what didn’t make sense because of my limitations rather than my knowledge. I didn’t even have those buckets to work with.

Among the things that made less sense to me were people, especially in groups and organizations. I understood classrooms, I didn’t understand recess.

My intuitions about the logical, systemic, and structured world developed and improved pretty robustly. My intuitions about the human world were weaker and slower to develop.

One choice would have been to stay within the bounds of the logical. But my innate curiosity kept pushing me to make sense of all of my environment.

There’s a long-running split between the human and the systemic. C.P. Snow called it “The Two Cultures” back in 1959. Working that gap has been the place that suits me, although I prefer to work it at a more detailed level than Snow. Recognizing a big problem doesn’t yield a big solution. It calls for building bridges one piece at a time.

Learning and knowledge management in start up mode

 Becoming a founder of Diamond Technology Partners in 1994 was a textbook example of leveraging your network. In late 1993 I was ready for something new. I had published my first book and sent a copy to a former boss and mentor with a note that I would like to use his name as a reference as I started to look for that next new thing. Our phone rang early on a Saturday morning in October—this was when phones were still attached to walls. My wife answered and handed the phone to me.

It was Mel. He had read the book the night before after traveling all week. He quickly came to the point. We needed to meet as soon as we could synchronize our travel schedules. I could not use his name as a reference.

Within the week we were seated in a conference room at the Admiral’s Club at LaGuardia Airport. Mel shared one of the very first copies of the Diamond business plan. The plan included a projected organization chart with a box labeled “Chief Knowledge Officer/Chief Learning Officer.” The only job Mel was prepared to help me land was to figure out how to make that box work.

Three months later I began commuting from Boston to Chicago. By June my wife and two young boys were in a new house on the North Shore of Chicago trying to make new friends while I tried to figure out what that box might mean.

Putting knowledge management and learning on our to do list demonstrated no special insight; these were issues that all professional services organizations were tackling. Professional services firms live off their accumulating knowledge and their ability to disseminate that knowledge across the organization. Knowledge management and learning are core requirements.

Drawing different maps

What was less clear at the time, to me at any rate, was that the lessons from established firms could not simply be transplanted. The issues and problems they faced were problems of maturity; their solutions addressed their problems. We had to work out how our problems were different before we could begin to design appropriate solutions.

I think of this as drawing different maps because it reminds me that we were in different territory. That wasn’t immediately obvious. We had all come from established firms and had limited experience in start up environments. It was tempting and comforting to view our challenges as simply matters of resources and scale.

The limited resources of being a start up turned out to be an advantage. We had combined knowledge management and learning in one box on the organization chart because we didn’t have two people to warrant two boxes.

We did not have a “Diamond” way of working. We were actively creating one out of the McKinsey/Accenture/Booz/Bain/Ernst&Young/IBM/LEK ways we drew on from our collective experience. Our clients worked with us because they had problems that didn’t fit any of those established ways.

There’s the business you plan and there’s the business the market tells you that you are in – you had better learn to listen to the market. Rather than match problems to the solutions we understood from experience, we let the features of problems shape our approaches. Our learning and knowledge problems were tightly coupled.

Our knowledge management problem was not about how to codify and scale our work so that it could be reliably distributed. It was to push on the boundaries of what we knew to make room for more new ideas. Our fundamental knowledge management problem was how to extract and organize insights from the innovation that was taking place. Our problem wasn’t so much knowledge management as it was knowledge creation.

We had to learn how to draw the maps we needed as we went further into the wilderness. Our problem was more like being part of a research team collectively trying to figure out what we were seeing, what it meant, and what to do about it.

Expanding the learning problem

With the market pushing the entire organization to create new ways to work, the scope of our learning needs expanded. Who had to learn, what they had to learn, and how they had to learn it all changed.

In established firms, the primary learning problem is to equip younger staff with the skills and expertise of their elders. For us, everyone had to deal with learning as both learner and teacher. Junior staff came wanting and expecting to learn; not all senior staff shared that mindset.

We also had to reduce the lag between knowledge creation where we learned something for the first time to sharing that learning with everyone else who could take advantage of it. In more stable environments, knowledge creation could be followed by investments in codification and confirmation. In our environment, we were happy if there were a handful of relevant examples from prior work that could help us shape what to do next.

Circumstance forced us not only into learning by doing but often learning while doing. There’s a rich base of knowledge backing the case for the power of learning by doing. We had the good fortune to be able to tap the insights of two of the leading thinkers in this area—Roger Schank and Alan Kay.

You can incorporate doing into almost any learning situation. Learning while doing turned out to be a bit trickier. Thinking in terms of how to better support reflective practice is one element. Building After Action Reviews into our projects was another.

We also deliberately blurred the lines between learning and practice. In our presentation skills class for new hires, rather than craft exercises we polled the field for research tasks and assigned those to students. If, for example, we needed to gather intelligence on a prospective client that became an assignment for a team in the course. Further, we brought in experienced staff from the field to evaluate and judge the quality of the research and the quality of the presentations.

The success of these experiments encouraged us to knock down more of the barriers between the field and learning/knowledge management. We brought experts back from the field to serve as instructors. We asked those same experts to ground the courses in their current client experiences.

What makes all of this matter beyond the challenges of one start up is that more of today’s organizations face similar problems of external environments that change too quickly to be captured in the knowledge management and learning practices of more mature organizations. We have all become start ups and our practices need to adapt to remain relevant.

McGee’s Musings turns 17

The experiment continues. The first post here came in 2001. This post is somewhere north of 1500. The number is inexact because portions of the earliest postings here aren’t easily tracked down.

The past year has seen a bit of a reboot here in terms of posting frequency., For those of you who’ve been following along, what would you like to see more of? What would you like to see less of?