The costs of context switching

Multiple ScreensMulti-tasking doesn’t work but our lives demand it anyway. This leaves us with the problem of how to compensate for the productivity and quality losses generated by work environments that demand parallel processing our brains can’t handle.

Why can’t our brains multi-task and what happens when we try? Left brain/right brain discussions aside, we only have one brain and that brain is single-threaded; it’s built to work on one cognitive problem at a time. Most of us can manage to walk and chew gum at the same time, but we can’t read the paper and discuss changes in the day’s schedule with our spouse simultaneously.

The bottleneck is attention. When we pretend to multi-task, what we are doing is cycling focus among the tasks competing for our attention. Each time we switch focus, we have to re-establish where we were in our work when we left off before we can begin moving forward. We also have to set aside the work we were doing along with whatever supporting materials we were using.

This process of redirecting focus is a context switch. Context switching is expensive because complex tasks—writing a blog post, debugging code, analyzing sales data—depend on equally complex mental scaffolding. When writing a blog post, for example, that scaffolding can include notes on the points to be made, memory of relevant previous posts ideas about upcoming blog posts, links and open browser tabs to supporting research, and so on. That scaffolding might be spread across multiple computer screens and program windows. It might also handwritten notes or paper copies of relevant supporting articles. All of that supporting scaffolding, along with the current draft of the blog post, helps you build up the mental structures that eventually lead to a finished draft of your post.

Suppose now that I need to put aside the blog post in progress to take an incoming phone call from my boss. It’s a call about a proposal we are putting together for a client. It might be just a simple call to confirm a detail in the proposal document, or it might be a more complex discussion about whether to rethink and reorganize the entire proposal. Regardless, I need to set aside the work on the blog post and flush my mind of all the details. I then need to call to mind the salient details of the client and the draft proposal as the call unfolds. In the first moments of this call, I’m not likely to be terribly articulate or smart. As the call progresses, I may need to call up various supporting materials and gradually fill in an entirely new context to contribute to the conversation.

Switching tasks means that you have to also break down one context and stand up a new context before you can actually begin to do any meaningful work. When the call is complete, you need to reverse the process to resume work on your blog post. Will you recall the insight that was just coming into focus when you were interrupted by that call from your boss? Or is it lost forever?

How expensive is this context switch? Research from the world of software development (see Jeff Sutherland’s Scrum: The Art of Doing Twice the Work in Half the Time) suggests that switching between two projects can result in productivity losses of 20%. Add a third project to your list and the costs rise to 40%. This means that each project gets no more than 20% of your attention and focus. Is it any wonder then that professionals work the hours that they do?

Step one in solving any problem is recognizing it. Limiting the number of projects you are working on and carving out big blocks of time to focus exclusively on each project helps. This is the core advice of most time management gurus. Few of us, however, have that much control over our responsibilities. A more attractive target then is to think about ways to lower the costs of context switching. We’ll come back to that in the next post.

Knowledge management matters more to you than to your organization

I gave a talk on Saturday for ChicagoLand PMI about why knowledge workers needed to develop strategies and the supporting habits and practices to manage and develop their know how across organizations and across time. If you’re interested you can find a copy of my slides on Slideshare.

Knowledge management as buzzword and practice originated in solving organizational problems. That’s where the big, obvious, problems are as well as the budgets. But the roots of the problem lie in the changing nature of work and careers at the individual level.

My father worked for three organizations in his career; I’ve worked for twenty so far and the number is likely to climb. Some might argue that this reflects either a severe case of ADD or a general inability to hold a job. Regardless, the trend is real; knowledge workers will work for more organizations and have shorter tenures at each. Organizations worry about the knowledge retention problems this creates; I’m more interested in the knowledge management problems it creates for individuals. I am aware of a handful of people who are also thinking about this; Harold Jarche, Luis Suarez. If you know of others, I would love to hear about it. 

The nub of my concern is this. You cannot rely on your memory and the experience it encodes. You also can no longer rely on having access to the institutional memory and artifacts of any one organization to supplement your limited human capabilities. You ought to be thinking about and planning for how you will accumulate knowledge and expertise over time. What personal infrastructure should you be building that can travel with you? How should you adapt your work habits and practices to simultaneously deliver value to your organization and enhance the value of your personal knowledge base? What new practices and skills do you need to add to your repertoire?

Entropy and knowledge management

What does the second law of thermodynamics tell us about knowledge management? There’s some pretty complex mathematics around the laws of thermodynamics, but the poet’s version will do for our purposes:

  1. You can’t win
  2. You’re going to lose
  3. You can’t get out of the game

Life is a constant battle against entropy or disorder. Cars break down; they don’t repair themselves. Left to themselves, files, books, and ideas become disorganized. Organizations and the knowledge workers inside them are engaged in a constant, but doomed, fight against entropy; the order they bring is always temporary.

Knowledge management is one of many disciplines engaged in that fight. If entropy is destined to win, what does that tell us about how to carry on the fight?

It reminds us that perfection is the wrong goal. You can’t define a perfect taxonomy; 100% compliance with the documentation standards is wasted effort; there will always be something more pressing than the paperwork. This matters because the personalities attracted to the apparent orderliness of knowledge management tend to be seekers of this impossible perfection. You want to temper that predisposition, not feed it.

Surrendering to disorder isn’t a good strategy either. Let the reality of entropy shape our strategies and practices. There are things we should worry less about and things we might better do differently.

We should worry a lot less about perfection and completeness and strive instead for a standard of “good enough”. That requires more judgment and sensitivity to unique circumstance than most organizations—and many individuals—are comfortable with. Black and white makes for easier, albeit impossible, compliance standards and management.

If you are in a position to shift an organization in the direction of more gray, encourage that. If you are enmeshed in unrealistic organizational expectations, strive for only as much compliance as will keep the auditors and censors at bay. I’m not advocating open rebellion, or even mild “civil disobedience”; simply be comfortable that you have the laws of physics on your side while you quietly ignore stupider requirements.

If entropy is the law, how might you operate differently?

Learn where small efforts now postpone or eliminate major remediation efforts. Whatever you opt to do now, you are going to live with that choice later. Make your choices with that appreciation of a disorderly later. You are never going to go back later to add the appropriate tag, improve the name of the file, or reorganize the project team’s directories. Recognize the places and moments where a tiny injection of order now will pay lasting dividends. Don’t pretend that you can get organized after the press of the immediate has passed.

Entropy is inevitable. As a knowledge worker, your task is to create pockets of order out of the noise. As you create those pockets, don’t increase the noise everywhere else.

EntropyAndKidsHMP Comics

Carve out time and space for deep thinking

 

Deep Work: Rules for Focused Success in a Distracted World, Cal Newport

If your value to an organization depends on the quality and insight of your thinking, Cal Newport’s latest book, Deep Work, offers important insights about how to think about your thinking. The forces at work in our environment and in our organizations favor quick, shallow, and social over other forms of thought. That is generally adequate for much of the activity that fills our days.

Exceptional value, however, finds its roots in sustained, focused, individual thinking and reflection. Deep Work builds the case for this mode of thinking and offers paths to carve out the necessary time and develop the necessary mental muscles to engage in deep work more intentionally and predictably.

Newport defines deep work as

Professional activities performed in a state of distraction-free concentration that push your cognitive capabilities to their limit. These efforts create new value, improve your skill, and are hard to replicate.

The ability to think deeply is a skill, so it can be developed. Much of the book offers counsel on techniques and practices that can help you develop your skills at deep work.

Newport is an academic computer scientist. In his world the contrasts between deep and shallow work can be stark. His binary distinction doesn’t transfer neatly to organizational settings where it is more useful to think in terms of a spectrum of work with shallow and deep anchoring the extremes. With the pressure toward superficial and shallow, there is great opportunity for individual knowledge workers to become more proficient at going deep.

Newport does offer practical advice for how to make deep thinking more possible. That advice needs tuning and refinement to work well in most complex organizations. Newport sums up why that effort matters thusly;

A commitment to deep work is not a moral stance and it’s not a philosophical statement—it is instead a pragmatic recognition that the ability to concentrate is a skill that gets valuable things done. Deep work is important, in other words, not because distraction is evil, but because it enabled Bill Gates to start a billion-dollar industry in less than a semester.

Practice and Performance

Cpl. Derek McGee, USMC MEU15 TRAP 2013

How do you strike an effective balance between practice and performance? In many realms we draw a distinction between performing and preparing to perform. Actors and musicians rehearse. Athletes practice. Soldiers train before they fight.

In other, equally demanding, realms the boundary is fuzzy; at times non-existent. Where does a sales rep or project manager practice? Where does a brand manager practice market segmentation? When does an investment banker practice designing a deal?

The notion of an apprentice observing and mimicking a master is one proven model that blends practice and performance. What troubles me is that this model works best when it is explicit. There needs to be some recognition that some performance settings are about both performance and practice; some fraction of your focus and attention needs to be tuned to learning.

My sense is that we have abandoned the notion of practice built into apprenticeship and favor performance exclusively. If we substitute performance only in place of practice and performance, do we abandon the possibility of achieving peak performance? How do we recognize situations that call for effective apprenticeship models? How do we design organizations so that they meet their performance goals and provide the necessary practice opportunities so that tomorrow’s organization can perform as well or better than today’s?

Connected Courses Course – An Experiment in Collaboration – #CCourses

I’m carving out time to participate in what I see as a worthy experiment in collaboration. It’s been organized by some of the most interesting people working on online learning and seems to be attracting an equally interesting collection of people interesting in participating.

Here’s what they have to say:

We invite you to participate in a free open online learning experience designed to get you ready to teach open, connected courses no matter what kind of institution you’re working in. We’ll explore how openness and collaboration can improve your practice and help you develop new, open approaches.

You can mix and match — take one unit or take them all, and go at your own pace. You’ll be joined by other participants from around the world who are looking to:

  • get hands-on with the tools of openness;
  • create open educational resources, curriculum and teaching activities and get feedback from a community of your peers; and
  • connect with and learn alongside other faculty, educators and technologists.

Sign up and receive updates from the organizers. Everyone is welcome, and no experience is required. We will all learn together in this free and fun opportunity to start planning your own connected course. The instructors, award-winning university professors from around the globe, are the innovative educators behind successful connected courses such as FemTechNetds106phonar, and the National Writing Project CLMOOC.

An orientation starts Sept. 2 and the first unit starts Sept. 15, 2014 and you can sign up and find more details about the topics we’ll be exploring at connectedcourses.net.

[Connected Courses Sign Up]

This is being billed as “a collaborative community of faculty in higher education developing networked, open courses that embody the principles of connected learning and the values of the open web.” I think it is something richer than that.

Paying attention is the least that you should do if you are interested in issues of collaboration, learning, and new organizational forms. Jumping into the pool with the rest of the crowd is a better idea.

Collaboration, games, and the real world

I’ve been thinking a lot about hard problems that need multiple people collaborating to solve. There’s no shortage of them to choose from.

This TED video from Jane McGonigal makes a persuasive case that I need to invest some more time looking at the world of online gaming for insight. Watch the video  and see if you don’t come to a similar conclusion.

 

Doing and Managing Knowledge Work: TUG2010 Keynote Reflections

I’m back from last week’s Traction User’s Group meeting, TUG2010, where Greg Lloyd graciously asked me to do the opening keynote. I’ve posted the slides on Slideshare and wanted to add some further commentary here.

First, one caution; when I do use slides I don’t design them to be standalone documents. There are too many bullet points in the world as it is. What I’d like to do here is highlight and elaborate on some of the key points I was trying to make.

Peter Drucker first called our attention to the importance of knowledge workers decades ago. The rest of us are slowly catching up to his ideas. One shift in focus that I’ve begun to emphasize is toward the knowledge work itself and away from the notion of knowledge worker as somehow distinct from other kinds of workers. Trying to distinguish who may or may not be a knowledge worker as opposed to some other kind of worker simply perpetuates pecking order games that do little to further the mission of an enterprise. We all do knowledge work  to some degree or another, we are all doing more knowledge work than before, and the important question is how to do that work more effectively.

The notions of visibility and observability have been central to my thinking for some time now. The evidence is clear that dealing with complex problems and thinking requires a certain amount of corresponding complexity and mess in our working environments. To those whose focus is on stability and operational control, mess, of course, is disturbing. So disturbing that we ridicule those who deviate from the presumed ideal. We do so at a greater organizational cost than we realize, however, when we ignore the complexity in the environment that is driving the mess.

I introduced the following simple map to suggest just how unavoidably messy the real world of knowledge work can be. The x-axis maps the inherent structure of the knowledge "stuff" we encounter; the y-axis maps the degree to which knowledge stuff is individual or social. It didn’t take long to identify a multitude of items and objects that you might routinely encounter as you go about your work.

KnowledgeStuffMap-2010-10-19-1045

It’s tempting to simplify this reality in some way. Many years ago P&G was famed for teaching its managers to distill their arguments into one-page memos. Too many consultants and speakers opt to squeeze all of their output into slideuments; which merely transfers the problem somewhere else. Senior executives rely on staffs to filter the stream at the risk of filtering out the essential insight or data point that truly informs.

The strategy I prefer is to accept the fundamental messiness and seek ways to tame it enough to make it manageable. Part of that relies on exploiting the natural pattern-seeking, pattern-matching capabilities of the human mind. Part relies on enlisting the pattern management capabilities of the other human minds in the system to supplement your own capacity. Both of which also need to be tempered by appreciation for the limits of those same capabilities.

Taming the mess breaks into three layers of practices:

  1. Hygiene. The proliferation of objects in a physical office offer a host of clues about their contents and relative importance; size, shape, color, location on a shelf or desktop, position in a stack, etc. In a digital environment you need to provide the equivalent of those clues explicitly and consciously. Seemingly mundane decisions about the file names you choose, for example, can make large differences when you are later scanning through a page of search results. Most of today’s systems provide little real assistance in this arena; you and your teams need to develop their own standards for naming files, managing versions, and other details of the knowledge stuff they work with.
  2. Metadata. i wish there were a more homespun term for this layer. One of the central tricks to taming the flood of data and information that constitute your digital world is to add more data to the flood. The ability to tag the items you create or encounter with labels that are meaningful to you greatly leverages the other tools at your disposal. Merge those tags with the tags of those in your social network and you shorten the path to finding what you are searching for still further, either on your own or through your network’s help.
  3. Context. One of the least appreciated aspects of messiness in the physical world is the context it provides. There’s a story attached to each pile and object; a story that can be triggered by its context. The power of this context is why students do better on tests when they take them in the same classroom they took the course in than in a random room. A filename in a directory listing or a document displayed on  a monitor lack this ready context and are poorer for it. The alternative is to become more mindful of the importance of context and make an effort to capture it explicitly and contemporaneously. This is rationale behind such notions as narrating your work, and developing a digital portfolio.

There is a payoff to all of this for both individuals doing knowledge work and the organizations they contribute to. Once again Peter Drucker said it first; "the most valuable asset of a 21st century institution will be its knowledge workers and their productivity."  Economic growth in this century depends on our ability to improve knowledge work productivity; until you can see it, you can’t improve it.

How better thinking about deliverables leads to better knowledge work results

Deliverables should be a knowledge worker’s best friend but, like all good friendships, the relationship can get complicated. One of the most direct ways to improve the quality of knowledge work is to spend more time and effort to define appropriate deliverables.

My faith in deliverables has grown out of decades of practical consulting experience, where defining the appropriate deliverable can mean the difference between mediocrity and success. A deliverable is an identifiable work product intended for someone outside the knowledge work process that marks the end of that particular process. Examples of deliverables include:

  • A consulting report prepared by the consultant or consulting team containing recommendations to the client sponsor.
  • A list of recommended job offers to prospective employees prepared by a campus recruiting team and handed to the hiring manager.
  • Production Release 1.0 of a new application system installed and operating on the companies intranet.
  • A project work plan and business case delivered to a steering committee for approval.

Why Deliverables Are Relevant

In the world of industrial work, process outputs are well-defined. This gives organizations a straightforward process improvement path. Holding the outputs constant allows process improvement to proceed by identifying and eliminating activities that don’t add value to the output; to break down, reorder, and redesign process steps to create more standard outputs with less input; and so on.

This path isn’t available to knowledge work practices. The goal is not to produce well-defined, standardized outputs. Deliverables are the closest analogue but their value depends on how well they match the unique needs of their users. No one is interested in a spreadsheet full of someone else’s data; no teacher is likely to value a copy of a paper you’ve submitted to another class. Understanding what aspects and features of a knowledge work deliverable are most valuable to its intended user is key to focusing efforts on producing the desired deliverable.

Natural vs. Artificial Deliverables

Some knowledge work processes produce obvious and natural deliverables. A campus recruiting effort generates a list of candidates to extend offers to. There may be a variety of associated supporting documents (e.g., interview notes, assessments), but the list is the deliverable and is a natural outgrowth of the process.

Other knowledge work processes generate more amorphous or less obvious outputs. For example, what’s the deliverable for a typical project status meeting? Shared understanding among the project team? Agreement about the state of the project between the team and the project sponsor?

Consider the typical status report as a more artificial than natural deliverable and ask how much value resides in this particular item. If you can define the deliverable more effectively, is the world a better place?

Defining Better Deliverables

The better we can define deliverables, the better and more effective we can make our knowledge work. Rather than ignore the end product, we need to be systematic in extracting as much as we can about the expected deliverable that can guide our effort to create it. There are three productive paths to explore in using the deliverable to drive your knowledge work.

  • Path #1: Understand the user’s essential quality need

    Does this deliverable need to be exactly correct whatever the time or cost, as good as possible given a particular deadline/budget, or the best you can come up with on the phone? Each deliverable demands a different approach; each requires a conversation (and perhaps a negotiation) with the end user. The history of systems development is littered with examples of failing to follow this path.

  • Path #2: Balance uniqueness and uniformity

    We’ve been conditioned by one hundred years of industrial experience to value uniformity. In knowledge work, the value of uniformity is to free time and attention for the essential uniqueness we’ve defined. The folks in marketing may believe that corporate identify standards are about branding. For a knowledge worker, they let you ignore formatting decisions to focus on the content that matters.

  • Path #3: Specify stopping conditions

    There was an joke that was old even in my early consulting days that you knew the design phase was complete when the budget ran out. Possibly the biggest challenge for managing knowledge work is determining how to recognize that the deliverable is done. Think of the unfinished doctoral dissertations and manuscripts of the great American novel gathering dust on a shelf or lost in a lonely directory of a disk drive. Too many knowledge-work efforts fail simply because no one thinks about how to recognize what "done" will look like.

Focusing on deliverables means working backwards with the end in mind. Knowledge work is valuable to the extent that it produces end results, i.e. deliverables, that meet the unique requirements of a particular customer or end user. Time invested in understanding what that deliverable should look like will yield the greatest return in defining and focusing the activities that contribute to creating that particular deliverable at the right time and place.

Finding knowledge work practices worth emulating and adapting

[Cross posted at FASTForward Blog]

How might we best go about improving knowledge work, both practices and outputs, in today’s complex organizational environment? Are there paths other than simple trial and error that might lead to systematic gains?

Frederick Taylor and his followers built their careers on finding the one best way to carry out a particular physical task. Later proponents of this way of thinking transferred their approach to defining the one best way to carry out information processing tasks. John Reed of Citibank launched his career by applying factory management principles to automating check handling. Reengineering essentially rebooted these approaches for a richer technology environment, but held to the premise that outputs were a given, tasks could be well-defined, and processes could be optimized.

Peter Drucker, in his typical way, pointed out that the key to understanding and improving knowledge work (PDF) was that there was no defined task to be optimized. Knowledge workers start by defining the task at hand and an output of suitable quality. This is not an approach which lends itself to conventional improvement or optimization approaches.

Two useful approaches come to mind. One would be to identify and shadow individual knowledge workers deemed to be particularly effective. Observing, understanding, and emulating their personal practices would be time well spent. A second approach would be to identify a class of knowledge workers who have been dealing with the problems of knowledge work in the modern enterprise long enough to have developed practices and approaches that might be broadly adaptable to knowledge work activities in general.

Both of these approaches are well worth undertaking. Today, I’d like to take a look at this second approach. A recent blog post by Eric Raymond prompts looking at a group I’ve often thought of as the leading edge of modern knowledge work– software developers. Over the last half-century, this group has been inventing and developing the technological infrastructure that shapes our modern enterprises. As such, they have been the first to encounter and address the challenges of knowledge work. Certainly any group responsible for the Internet and the invention of open-source software will have lessons for the rest of us as we try to bring forth our own examples of knowledge work products.

Raymond, among many other things, is the author of the excellent The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. He also maintains the always interesting and provocative blog Armed and Dangerous. In a recent post, “the social utility of hacker humor”, Raymond dives into the number of the behavioral norms that he believes characterize hackers and software developers. The entire blog post is well worth your time, but let me call your attention to the following excerpt:

…every once in a while something erupts out of them that is a game changer on a civilization-wide level. Two of the big ones were the Internet and open-source software. These two movements were intimately intertwined with hacker culture, both produced by it and productive of it. The origins of our tribe go back a bit further than either technology, but we have since re-invented ourselves as the people who make that stuff work.

And I don t mean make it work in a narrow technical sense, either. As long as there are people who laugh at INTERCAL and RFC1149 and the Unix koans of Master Foo, and recognize themselves in the Jargon File, those same people will care passionately that computing technology is an instrument of liberation rather than control. They won t be able to help themselves, because they will have absorbed inextricably with the jokes some values that are no joke at all. High standards of craftsmanship; a subversive sense of humor; a belief in the power of creative choice and voluntary cooperation; a spirit of individualism and playfulness; and not least, a skepticism about the pretensions of credentialism, bureaucracy and authority that is both healthy and bone-deep.

[Raymond, Eric S. Armed and Dangerous. “The social utility of hacker humor“]

Substitute “knowledge worker” for “hacker” and I believe we will find parallels worth exploring.

This is still in the working hypothesis stage. I can think of a number of practices that might prove worth emulating and some useful entry points into learning more.

Practices worth investigating

Serious software developers have adopted a number of practices as they’ve struggled with the challenge of designing, developing, and evolving products that are pure thought stuff. To the extent that knowledge work is also a process of developing outputs that are themselves largely thought stuff, these practices ought to have analogues. Here’s a preliminary list, in no particular order.

  • Version control/source code control. Final outputs and products grow through a process of successive refinement.