Repeatable Processes and Magic Boxes

There is a trap hidden in most efforts to create repeatable processes and systems that try to guarantee predictable results in doing knowledge work. To avoid that trap, you must learn to recognize and manage the magic box.

The promise of repeatable processes is to identify, design, and sequence all of the activities that go into producing a specified output. It is the core of industrial logic. For industrial logic to turn out uniform results with predictable quality, craft must be transformed into proven systems and repeatable processes. Variation must be rooted out and eliminated. To do this, all of the design thinking necessary to produce the product must be extracted from the process and completed before any further work can be done. You get the prototype product done right and then you churn out a million replicas. For this strategy to work, all of the magic must take place before the first process step ever occurs.

Knowledge work cannot be forced into an industrial straightjacket. The essence of knowledge work is to produce a bespoke result. I have no interest in the strategy McKinsey developed for its last client; I want the strategy that applies to my unique situation. I do not need the accounting system tailored to GM’s business; I need an accounting system matched to my organization. And therein lies the rub. I may want a bespoke accounting system, but I’m not willing to pay for it. Forty years ago, in fact, I had to pay for one anyway because off-the-shelf accounting software didn’t exist. All software development was custom; as was all consulting work.

Consultants and software developers are not stupid people. They could see that, despite the necessity of producing a unique product at the end, much of their work had elements of the routine and predictable. To increase the quality of their work, to train new staff, to improve their economics, and to better market their cumulative experience, knowledge work organizations worked to transform their practices into repeatable processes and methods. New industries were created as software developers redesigned their code to segregate what needed to be customized from what constituted a common core.

In this effort to apply industrial logic to what was fundamentally creative work, most organizations were sloppy or short-sighted about managing what was a fundamental tension between industry and craft. What was big, and shiny, and marketable was the packaging of cumulative experience into a consulting methodology, or a software development process, or a customizable software product.

What could not be eliminated, however, was the essential craft work necessary to employ the methodology or to customize the product. What happened was that this craft work was pushed into a box somewhere on the process map or into a line item in the standard workplan.

This is the “magic box.” Whatever its name or label, it is the step where the necessary creative work takes place. This is work that cannot be done until the moment arrives.

Why does it matter to identify which boxes in the process require magic? Because they determine the quality of the final result. The other boxes only matter to the extent that they set you up to succeed in the magic box.

How do you recognize the your are dealing with a magic box? What are the clues that differentiate it from among all the other boxes? Sometimes you must approach this by process of elimination; many boxes—“conduct field interviews”, for example—are more easily identified as not possibly containing magic. As for positive identifying features, look for language that suggests design thinking steps or analysis that isn’t tightly specified. “Develop market segmentation approach”, or “design chart of accounts” are examples of possible magic boxes.

Understanding which boxes are which in a process is essential to managing the process effectively. Regular boxes can be estimated and managed more tightly than magic boxes. Data collection, for example, is usually straightforward; analyzing it, requires more flexibility and adaptability. Collapsing these related, but distinct, activities into a single step would be a poor project management decision. Just because creative work cannot be controlled in the same predictable way as industrial work does not relieve managers of their responsibility to make effective use of limited resources. Being clear and isolating the magic boxes from the ordinary ones is essential to making those resource deployment questions.

Repeatable processes are often marketed and sold as “proven approaches” that eliminate the trial and error that the uninitiated risk if they strike out on their own. This has enough truth to be dangerous. Traveling in new terrain is safer with an experienced guide. The guide may help you get to where the underlying geology is promising, but cannot guarantee that you will strike gold. Honest guides will emphasize the distinction. But it is a distinction that is only meaningful to those who can hear it.

Trial and error is an unavoidable feature of creation. A serendipitous error is often the seed of a creative solution. Understanding where the magic needs to occur helps you distinguish between unproductive and potentially productive error. Unproductive error is an opportunity for learning and process improvement; it should be carefully reined in. Potentially productive error must be permitted and encouraged. That can only be done effectively by understanding where the magic needs to occur.

Some related links:

• Knowledge work as craft work

• Enterprise Systems \| Crafting Uniqueness in Knowledge Work

• Managing the visibility of knowledge work