Cognitive surplus and organizational slack

[Cross posted at FASTforward]

Clay Shirky’s got a new talk and he’s taking it on the road. It’s stimulating a good bit of thoughtful discussion around the web. Here’s a video version of his talk:

Shirky has also posted a transcript of the talk on his site, if you’d prefer to read instead of watch. The talk is a riff on one of the themes of his new book, Here Comes Everybody: The Power of Organizing Without Organizations. I’ll post a complete review of that shortly; it’s well worth you’re making time to read it.

One of the stories Shirky hangs his argument on is an interchange with a TV producer about the creation and growth of Wikipedia. Here’s how he tells it:

I started telling her about the Wikipedia article on Pluto. You may remember that Pluto got kicked out of the planet club a couple of years ago, so all of a sudden there was all of this activity on Wikipedia. The talk pages light up, people are editing the article like mad, and the whole community is in an ruckus–“How should we characterize this change in Pluto’s status?” And a little bit at a time they move the article–fighting offstage all the while–from, “Pluto is the ninth planet,” to “Pluto is an odd-shaped rock with an odd-shaped orbit at the edge of the solar system.”

So I tell her all this stuff, and I think, “Okay, we’re going to have a conversation about authority or social construction or whatever.” That wasn’t her question. She heard this story and she shook her head and said, “Where do people find the time?” That was her question. And I just kind of snapped. And I said, “No one who works in TV gets to ask that question. You know where the time comes from. It comes from the cognitive surplus you’ve been masking for 50 years.”

So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project–every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in–that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it’s a back-of-the-envelope calculation, but it’s the right order of magnitude, about 100 million hours of thought. And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that’s 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, “Where do they find the time?” when they’re looking at things like Wikipedia don’t understand how tiny that entire project is, as a carve-out of this asset that’s finally being dragged into what Tim calls an architecture of participation. [Gin, Television, and Social Surplus]

The notion of “cognitive surplus” is a clever and useful way to frame the issue. Now, Shirky is primarily interested in the societal level impacts of new technologies. Big numbers help his argument tremendously, but they are a little bit like the arguments for why you might want to target your new consumer product at China (“if we only get one person in a hundred to drink our new sport drink, we’ll sell millions!”). Or the dotcom era arguments for capturing eyeballs. I don’t think that Shirky falls into this trap himself. Here, and in his book, he explicitly talks about how the design and architecture of systems such as Wikipedia leverage cognitive surplus in granular ways to exploit these large numbers.

My primary interests are inside organizations. How can we translate and adapt these insights into those environments? Organizational theorists, not being as clever or market oriented as Shirky, did not think up a notion as attractive as “cognitive surplus.” Instead, they talk about the notion of “organizational slack.” In hindsight, a very poor choice of words. For the last two decades, or more, organizations have been rooting out “slack” wherever they could find it. When the goal is efficiency, this is an appropriate strategy. However, it leaves no capacity for innovation and adaptation. Those few organizations that explicitly provide this capacity, such as Google’s 20% rule, are deemed notable and newsworthy.

The first order of business for business is to immediately appropriate Shirky’s term. Organizations that care about innovation and adaptive capacity should begin talking about “cognitive surplus.” Look for ways to measure it, if only crudely, and increase it.

The second task is to better understand and appreciate how various new technologies and tools let organizations derive benefit from smaller grains of cognitive surplus. Google’s 20% rule is a product of a time largely before blogs and wikis. Can an organization combine the tools with a one hour or 10 minute rule? Can we get value out of an hour a week or 10 minutes contributing to an internal wiki? Clearly, we will need to design some thoughtful support and encouragement processes around the tools in order to take advantage of a different scale of participation.

The third task is to monitor how well the large number phenomena outside the enterprise operate inside. We may discover critical mass issues; efforts below a certain scale are doomed to fail, while slightly larger efforts will need an extensive “life-support” system to survive. Other efforts may need support scaffolding but can become self-sustaining. Today, we have far more questions than answers. Shirky has provided us with some good new notions to start finding answers. I’d also recommend some of the following discussions that I’ve come across so far:

Designing with “harmless failures” in mind

Ed Felten at Freedom to Tinker has some interesting points to add to Bruce Schneier’s piece on “Security Mindset” that I posted about yesterday. Felten focuses on the notion of “harmless failures.” It provides still more reason to approach all systems design problems with an eye firmly fixed on the social context in which your technology will operate.

The Security Mindset and “Harmless Failures”

…Not all “harmless failures” lead to big trouble, but it’s surprising how often a clever adversary can pile up a stack of seemingly harmless failures into a dangerous tower of trouble. Harmless failures are bad hygiene. We try to stamp them out when we can.

To see why, consider the donotreply.com email story that hit the press recently. When companies send out commercial email (e.g., an airline notifying a passenger of a flight delay) and they don’t want the recipient to reply to the email, they often put in a bogus From address like donotreply@donotreply.com. A clever guy registered the domain donotreply.com, thereby receiving all email addressed to donotreply.com. This included “bounce” replies to misaddressed emails, some of which contained copies of the original email, with information such as bank account statements, site information about military bases in Iraq, and so on. Misdirected ants might not be too dangerous, but misdirected email can cause no end of trouble.

The people who put donotreply.com email addresses into their outgoing email must have known that they didn’t control the donotreply.com domain, so they must have thought of any reply messages directed there as harmless failures. Having gotten that far, there are two ways to avoid trouble. The first way is to think carefully about the traffic that might go to donotreply.com, and realize that some of it is actually dangerous. The second way is to think, “This looks like a harmless failure, but we should avoid it anyway. No good can come of this.” The first way protects you if you’re clever; the second way always protects you.

Which illustrates yet another part of the security mindset: Don’t rely too much on your own cleverness, because somebody out there is surely more clever and more motivated than you are.

Designing with failure in mind

Bruce Schneier is high on my list of smart people to pay attention to. His blog, Schneier on Security, always provides useful insights into the interplay between technology and people. Yesterday, he offered an interesting observation about what he labels “the security mindset.”

Schneier on Security: The Security Mindset.

….

Security requires a particular mindset. Security professionals — at least the good ones — see the world differently. They can’t walk into a store without noticing how they might shoplift. They can’t use a computer without wondering about the security vulnerabilities. They can’t vote without trying to figure out how to vote twice. They just can’t help it.

SmartWater is a liquid with a unique identifier linked to a particular owner. “The idea is for me to paint this stuff on my valuables as proof of ownership,” I wrote when I first learned about the idea. “I think a better idea would be for me to paint it on your valuables, and then call the police.”

Really, we can’t help it.

This kind of thinking is not natural for most people. It’s not natural for engineers. Good engineering involves thinking about how things can be made to work; the security mindset involves thinking about how things can be made to fail. It involves thinking like an attacker, an adversary or a criminal. You don’t have to exploit the vulnerabilities you find, but if you don’t see the world that way, you’ll never notice most security problems.

I’d push his observations a bit farther. When you are designing and building systems that incorporate people and technology, you had better think about both how to make things work and about how things might fail.

Human systems are interesting and effective because they are resilient. Good designers allow for the reality of human strengths and weaknesses and factor both into their designs. Too many poor or lazy designers ignore or gloss over failure modes. How many project plans have you seen, for example, that assume no one on the project team will ever be out sick? And then management complains when the project fails to meet its deadlines.

There’s actually quite a lot of good material on failure in human/technology systems and how to compensate for reality. I’d recommend the following as good starting points:

Tags:

Dealing with social in the enterprise

The theme at this year’s FASTForward conference is the “user revolution.” Don Tapscott gave a nod to Time Magazine’s selection of “You” as the person of the year in 2006 as part of his keynote Monday evening. References to Facebook, Flickr, and Wikipedia have been rampant throughout the general sessions and in hallway conversations. The question that remains unasked and, thus, unanswered is “how are things different inside the enterprise.”

One obvious difference is scale. Applications and services on the net have the entire population of net users to draw from. The 1/9/90 heuristic works nicely on the scale of the net as a whole. Inside the enterprise, the rule suggests that implementation efforts need to consciously manage participation and activity to compensate for the smaller population.

The second important difference arises from the need to manage participation within the enterprise rather than capitalize solely on “natural” participants. This collides with the aspects of the enterprise that substitute artificial order for natural order. Large-scale enterprises explicitly design roles and responsibilities to address task requirements in a controlled fashion.

While organizational researchers and designers have been pointing out the limitations of control thinking for much of the last 20-30 years (if not more), the reality in enterprises is that control remains central to enterprise DNA. While insightful folks like Andrew McAfee identify the importance of emergence in the successful uptake of Enterprise 2.0 technologies, I think they tend to downplay the barriers created by the reliance on control. When McAfee talks about the importance of culture in successful Enterprise 2.0 efforts, he is fundamentally talking about enterprises that have managed to move past classical models of control. What makes this such a challenge is the extent to which these models are unarticulated or regarded as axiomatic.

The problem of emergence

Andrew McAfee’s Sloan Management article defining Enterprise 2.0 is available for download, so I took the opportunity to reread it, after a recent chat over coffee with Jordan Frank of Traction Software.

Enterprise 2.0 is Now Free

The article, at least.  MIT Sloan Management Review, with support from IBM, is making a set of ‘classic’ (thanks!) articles freely available to all comers. So the full text of my original SMR article “Enterprise 2.0: The Dawn of Emergent Collaboration” can be downloaded here.

I don’t know if this is a temporary or permanent arrangement, so I’d suggest acting quickly.

One of McAfee’s central arguments is on the importance of emergence in successful Enterprise 2.0 initiatives. Here’s the way he puts it:

Second, the technologists of Enterprise 2.0 are trying hard not to impose on users any preconceived notions about how work should proceed or how output should be categorized or structured. Instead, they’re building tools that let these aspects of knowledge work emerge.

This is a profound shift. Most current platforms, such as knowledge management systems, information portals, intranets and workflow applications, are highly structured from the start, and users have little opportunity to influence this structure. Wiki inventor Cunningham highlights an important shortcoming of this approach: “For questions like ‘What’s going on in the project?’ we could design a database. But whatever fields we put in the database would turn out to be what’s not important about what’s going on in the project. What’s important about the project is the stuff you don’t anticipate.” [p.25]

In an accounting or ERP system, the system’s designers specify all aspects of workflow, database design, and information structure in advance. Users are expected to select from among pre-defined choices and enter only such data as the designers have provided for. In designing a system for emergence, the designers leave a number of these decisions open; waiting for users to fill in the blanks. So, for example, instead of locking down a taxonomy for categorizing documents, the designers might provide a tagging system to allow a folksonomy to emerge from the idiosyncratic choices of each user.

The attraction of emergence is twofold. One is the realization that conventionally structured approaches have generally failed when tackling knowledge intensive problems. Knowledge work and knowledge workers don’t mesh well with the structuring techniques appropriate to industrial work.

The second is the perceived success of emergent approaches behind current Web 2.0 success stories on the Internet. It’s easy to see the power of emergence in such examples as flickr, facebook, and technorati.

Transplanting those experiences inside the boundaries of the organization is no simple task. What works at the scale of the public internet may not generate sufficient momentum within the confines of a single organization. Moreover, Internet success stories ignore or gloss over the failures and also-rans. Failure in the market is tolerated in ways that don’t translate well inside organizations. 

You want the energy and creative outcomes that can come from a successful emergent approach, but you can’t simply rely on unaided market forces to fuel the process. “Unaided” is the key notion. Emergent successes in the market benefit from scale and viral strategies, but they don’t happen by accident. For starters, there is a marketing strategy and plan that exists in parallel with a technology implementation plan.

Enterprise 2.0 efforts within organizations also need a marketing plan to accompany their implementation plans. Like any marketing plan, this plan must identify and characterize its target market of potential users. In particular, the plan needs to identify those potential users who are most likely to benefit from the new capabilities and whose successful use of the technology will be interpreted as an endorsement to be emulated.

Is a marketing plan, by itself, sufficient to allow the other aspects of an Enterprise 2.0 implementation to emerge from use? Appropriate scaffolding and careful seeding of content will prove more useful. A complete taxonomy, for example, may overwhelm a small set of potential early adopters. On the other hand, an empty tagging system will prove too much of a blank slate for users more accustomed to the structures of conventional systems. Providing a sample of suggested tags or categories coupled with some live content can point users in the right direction.

Supporters and early adopters will also benefit from coaching and mentoring on how to use selected technologies to accomplish their goals. This coaching would focus on working out strategies for how to use the technology to accomplish specific business and organizational goals. This requires a different kind of engagement between the implementation team and the target user group. In particular, it entails introducing the user population to key design questions and issues that would typically have been dealt with by the implementation team.

In some respects, “emergence” is a fancy organizational development word for “messy.” The more our systems must deal with the complexities of the real world, the messier they must be to accommodate that messiness. Large scale organizations in general, and IT organizations in particular are not generally comfortable with messiness. Calling it emergence helps. But the fundamental need is to acknowledge that it is more useful to learn as we go and build our systems accordingly, than it is to force fit these systems into structures that cannot contain them.

 

 

Research on business collaboration from IBM

James Robertson pointed to this last month. It is one of several excellent articles in an issue of the IBM Systems Journal on the topic of business collaboration. While the writing is a tad dry, the thinking and the research is nicely grounded in some real data for a change.

Beyond predictable workflows: enhancing productivity in artful business processes

C. Hill, R. Yates, C. Jones, and S. L. Kogan have written a journal article on managing ‘artful’ processes. To quote:

Aside from the issues of scale, lock-in, and dependency, certain types of work simply cannot be formalized well enough to safely entrust to an enterprise application. The goals and methods of some processes change too quickly over time; for example, the process of designing high-technology products. In some processes, it is primarily the content in each process instance — rather than the process itself — that determines the outcome; for example, a request for proposal (RFP) process. Most important, many highly specialized processes are developed or refined locally at the individual or small-team level such that the process cannot easily be separated from the specific people who perform it; for example, managing client relationships in professional services firms. While the framing process may be stable at an abstract level, the key details are not. They depend on the skills, experience, and judgment of the primary actors. We denote these kinds of processes artful in the sense that there is an art to their execution that would be extremely difficult, if not impossible, to codify in an enterprise application.

[Thanks to Martin White.]

Euan Semple on strategies for implementing Enterprise 2.0

Insightful advice from Euan. I’m not sure, however, that most organizations can avoid the temptation to meddle and manage this instead. That will slow adoption down in most cases.

The 100% guaranteed easiest way to do Enterprise 2.0?

DO NOTHING

And then your bright, thoughtful and energetic staff will do it for you. Trouble is they will do it outside your firewall on bulletin boards, instant message exchanges personal blogs and probably on islands in Second Life and you will have lost the ability to understand it, influence it, and integrate it into how you do business.

The second easiest way is to find ways of allowing this to happen inside the firewall which can be as simple as sticking in some low cost or free tools and then making sure your existing organisation can:

GET OUT OF THE WAY

The third easiest way is to do the second easiest way and then engage those who would have done the easiest way and get them to help you:

KEEP THE ENERGY LEVELS UP

And the hardest way …….

…. you don’t need me to tell you that!

Technorati Tags:

Starting to unpack the promises of Enterprise 2.0

[cross posted at FastForward blog]

I was sitting next to James Robertson yesterday as Ray Lane delivered his opening keynote address at FastForward07. At one point James leaned over to me and joked that I was beginning to twitch. What was bothering me was that Ray was perpetuating the lazy and glib thinking of Nick Carr’s infamous Harvard Business Review article “IT doesn’t matter.”

Carr setup a series of artificial distinctions and misleading definitions to make the binary point that IT was always and everywhere a corporate utility function that had no place at the executive table. While that certainly played well in many organizations that had poured millions of dollars into ill-conceived technology initiatives, I doubt that it gained much traction in the board rooms at Amazon or Google. The question is not whether IT does or does not matter. The questions are how does it matter, when does it matter, how does it integrate with our broader strategic agenda, and what do we as senior executives need to understand about technology’s capabilities and possibilities in order to make intelligent decisions for our particular organization.

In the context of this conference, Lane is making a similarly broad assertion that “search is core.” That may make for an excellent marketing tagline, but I am struggling with just what it means for an organization. I am wary of any claims that boil down to “here is your silver bullet.” My own bias is that all of these promising technologies are just that-promising. Now, the task is to unpack and understand what it will take to turn the promise into reality in a particular organization at a particular time and place.

Business Problems and Root Causes

Thanks to Jon Husband for pointing this one out. I’ve been a fan of Evelyn Rodriguez’s Crossroad Dispatches for a long while. Her writing challenges me to take risks that I don’t always rise to, but always appreciate.

Business Problems and Root Causes

A delicious discovery whilst browsing this morning …

From Evelyn Rodriguez’ bio on her blog Crossroads Dispatches:

If one could honestly assess the root cause of many business problems – it’d be these intimately related concepts: being open is dangerous and being guided by the echoing fear in our heads is safe.

– Evelyn Rodriguez

Powered by Qumana

Tips for gaining adoption of Enterprise 2.0 technologies

[cross posted at FastForward blog]

We’ve been challenged to offer tips for gaining adoption of Enterprise 2.0 technologies by  James Dellow and James Robertson. Of the responses so far, I confess that I am most aligned with Euan Semple’s, who suggests that perhaps the call for adoption advice is premature. Here are some thoughts on adopting these technologies beyond the general strategies that apply to any collision between new technology and an organization.

  1. Start with your own learning. You are the target user base. Moreover, these are technologies whose value is not easily understood from casual use or from reading someone else’s account. Set up a blog and start keeping a daily journal with it narrating your work.
  2. Use Enterprise 2.0 tools to do your research on Enterprise 2.0 Get an RSS Reader and start subscribing to blogs talking about these technologies. Use your private blog to post items from your reader with your thoughts and reactions. Set up an account at del.icio.us and start using it to track your web surfing. Use a wiki to start organizing your research into a business case and plan for your organization.
  3. Find and enlist co-conspirators. Ignore the issue of resistance to organizational change. Route around it. Find a handful of other individuals you work with to join you in your efforts. If possible, include collaborators outside your organization as well. Examine and reflect on your struggles and mistakes as you learn.
  4. Ignore the IT organization or co-opt it. These technologies are inherently subversive to the established order of things in most organizations. Don’t fight it, exploit it. Start with services outside the firewall, unless you can find a sympathetic friend inside the IT organization who will help set up a sandbox server to play with. Don’t get caught up in trying to fit in with the existing technology architecture or standards. You’re initial objective is to understand how this class of applications interact with the business processes in your unique organization. Don’t fall into the trap of thinking the problem is about technology.
  5. Fix a broken process. Once you’ve developed some grounded experience with these technologies, you’ll be able to identify the process in your organization that can visibly benefit and that you have the power and authority to fix.  

Absent a specific organizational situation and a specific problem, I fear that most tips will, of necessity, be very generic.