Applying End-to-End Design Principles in Social Networks

Partial map of the Internet based on the Janua...

Image via Wikipedia

 Andy Lippman, at MIT’s Media Lab, offers provocative examples of learning how to think in network terms when designing services in a recent blog post from the Communications Futures Program at MIT. At the very heart of the Internet’s design is a notion called the end-to-end principle (pdf). The best network is one that treats all nodes in the network identically and pushes responsibility for decisions out to the nodes. Creating special nodes in the network and centralizing decisions in those nodes makes the network as a whole work less well.

In this essay, Lippman explores that notion by looking at examples of existing and potential services in telecommunications networks that could be improved by trusting the end-to-end principle more fully. Lippman takes a look at emergency services such as 911 calls in the US. As currently designed, these services allow individuals to reach a centralized dispatch center in the event of an emergency.

Emergencies are no longer solely about getting help for a fire or heart attack. Nor are they purely personal affairs, directed at or for a single individual. Consider the recent attempted attack on a Detroit-bound airplane where passengers provided the service (saving the plane). Early reports portrayed this as a fine solution. Indeed, there is discussion that the best result of increased airline security is that it has made people aware of the fact that they all have to pitch in to help when it is needed; they can no longer just rely on a remote entity a site to solve the problem for them.

End-to-End Social Networks
Andy Lippman
Fri, 01 Jan 2010 21:10:36 GMT

Lippman makes the point that we can benefit from thinking about ways to mobilize the network as a whole as an alternative to using it to direct messages to some centralized authority. Continuing to impose hierarchical notions on top of network designs risks missing other, potentially more powerful, options. We have a set of powerful new tools and ideas that we have yet to fully exploit.

The design reasoning that underlies the engineering of the Internet is applicable in organizational settings as well. Lippman’s examples are a good place to start in thinking how to apply them effectively.

Reblog this post [with Zemanta]

Does the CIO have a role in successful social media adoption?

Like everyone else who’s awake, my long-time friend and colleague Keri Pearlson and I have been trying to make sense out of the uptake of new "social" technologies into organizations. We are noodling on the hypothesis that the CIO represents the best choice if an organization wants to develop a social technology strategy that is both effective and reasonably efficient in the demands it exacts on the organization.

Saying the Dell or P&G has a social technology strategy is a common shorthand that obscures a more important truth. There are real people in specific roles who take on the responsibility for developing and deploying the collection of initiatives and programs that get labeled as an organization’s social technology strategy. The specific people and the particular functions involved greatly influence the success or failure of these initiatives

Some manager in marketing experiments with Twitter or a fan page on Facebook. A lawyer in the general counsel’s office raises a concern about whether an employee comment on Twitter creates a liability for the corporation. A divisional general who still has his assistant print out his email traffic creates a task force to develop a corporate social media policy proposal. While there may be no right answer for how an organization handles social media, these choices matter. The hypothesis that we are considering is this:

The CIO represents an excellent choice for who should coordinate an organization’s approach to social media/social networking.

Why we think this is a reasonable hypothesis

From an IT manager’s perspective, the technologies of social media/social networking appear quite simple. They are either web services hosted outside the firewall or they are very simple new capabilities hosted on internal servers. Compared with the complexities of a global ERP system, a distributed point-of-sale system, or a terabyte-scale data warehouse, social media/social networking capabilities are technologically trivial. Why then are they a problem relevant to the IT function? Why not simply let ownership and management of these capabilities reside in the business?

First, much of the value in social media/social networking lies in the masses of data they generate. Whether in the content of employees at Microsoft blogging internally or publicly about their work or in the network linkage data embedded in the interactions among customers and customer service staff using @ComcastCares on Twitter, there are masses of data to be managed and manipulated. IT knows and understands the issues that arise when dealing with data on this scale. Moreover, they understand how to filter through and extract insight from this data.

Second, there is huge potential value in connecting activity in social networking venues to specific business process steps embedded in the current enterprise support environment. This too constitutes an area where IT’s existing perspectives add value as social media/networking activity moves from experiment to operating at scale.

Third, many of the issues with social media/social networking cross functional boundaries in the organization. IT as a group routinely handles cross-functional issues in designing and deploying other technology around the organization. They will have established relationships with the right people around the organization and they will be sensitive to the kinds of organizational issues that arise in cross-functional undertakings.

The general point is that experiments with these technologies will occur naturally in multiple spots throughout the organization. As these experiments grow in scale and scope the particular management challenges that will appear fall squarely in the sweet spot of the IT function.

What we’re doing next

Organizational work is messy and complex. Social technologies are messy and complex. Put the two together and you have mess squared.

What that means is that there aren’t any maps and there aren’t any checklists. There is no cookbook or operating manual to follow. Not yet, at any rate.

The appropriate research strategy now is to capture and start to understand the messy stories of what is actually going on. It is too soon to strip the story down to its essentials, because we can’t yet differentiate critical step from colorful detail.

We are looking to develop case studies of what organizations are actually doing. At this point, it is premature to be distilling these stories into a coherent and over simplified narrative. For now, it is enough to get multiple stories of successful, failed, and too soon to tell efforts. Comparing and contrasting those stories will begin to reveal the patterns of what matters. if you’re interested, drop one of us a line or leave us a comment.

Socializing and knowledge management

Before Lotus Notes or SharePoint we had Happy Hour. Arthur Andersen/Accenture grabbed an early lead in knowledge sharing because it recognized the value of a liquor license long before there was even a technological environment capable of supporting the likes of Notes or SharePoint. Their efforts demonstrate why successful knowledge management is rooted in the social, not the technical. It’s a lesson we keep needing to learn.

I started my professional career in the consulting arm of Arthur Andersen & Co. years before the divorce that led eventually to the creation of Accenture. When I joined, the consulting group was known as "Administrative Services" and I spent a fair bit of time explaining to friends and prospective clients that I had nothing to do with office supplies or janitorial services. In those days, Andersen was justifiably known for its large investments in training its people in the skills and knowledge they needed to work effectively. It was one of the selling points that convinced me to join.

I joined Andersen shortly after it had invested in St. Charles. One of the primary training expenses was housing and feeding the hundreds of junior consultants being trained. It was the second largest expense item after the people costs themselves (both for instructors and for the students who weren’t generating revenue while being trained). Booking blocks of hotel rooms in Chicago or New York or London wasn’t going to be sustainable as Andersen continued to grow.

Just outside of Chicago, in St. Charles, was the campus of a recently failed Catholic girl’s college. Andersen’s partners bought the campus and transformed it into a center to house their training efforts. It made a great deal of sense and provided a smooth transition for all those young larval consultants; most of whom had just left similar campuses.

St. Charles was isolated and remote; an hour’s drive to the semi-bright lights of Chicago’s Rush and Division Streets. Few of us had the wherewithal to make the trip and courses were designed to provide little or no time to do so anyway. As a consolation prize, St. Charles had its own bar on campus. What the nuns who had run the college thought of this remains a mystery to me. What I have come to believe, however, is that getting this liquor license was the single most effective investment in knowledge management that Andersen ever made.

The bar at St. Charles was a safe place to share stories and a place where those with good stories mixed freely with those who needed to hear those stories. Better yet, the bar wasn’t a classroom. In a classroom, the teachers feel compelled to teach and the students feel compelled to feign wakefulness. In the bar, there was no teaching going on to interfere with the learning.

Before we started dressing things up in fancy terms like knowledge management and knowledge sharing we "talked shop." The bar was a natural place to talk shop. It was also a place where people came from all around the world. It was a place where we could start building the personal relationships on which future knowledge sharing would depend.

Today, of course, we operate in a more complex and widely distributed world. It can be harder to create and sustain the interactions needed to create those relationships. But keep that image of the bar in mind when you’re designing for your environment.

Emergent behavior and unintended consequences in social systems

One of the defining characteristics of Enterprise 2.0 implementation efforts according to Andy McAfee, among others, is the presence of emergent behaviors in the organization as participants interact with and adapt to new technology functions and features. The notion of ’emergent behavior’ is pretty well established in the study of complex systems. Yet it still seems to trouble many executives, particularly those with strong project management and operations backgrounds.

I was pondering this over the weekend and I think I’ve found a way to explain it in a more satisfying way.

Emergent behaviors are unintended consequences that make you happy.

We are social animals that have evolved to operate optimally in small groups (check out Dunbar’s number). As social systems get larger, they exceed our capacity to make accurate inferences and predictions. Complex organizations and political entities represent design solutions that compensate for these limits and allow us to take on tasks and efforts beyond the grasp of small groups. Technology adds to the complexity and increases the capacity of the system at the expense of making the system still more difficult to predict.

‘Unintended consequences’ is a consulting term for ‘oops.’ It’s a belated admission that it’s difficult to predict all the ways in which a system will react to its environment. A typical response is to work more diligently to lock things down, usually by squeezing out opportunities for human judgment and adaptability. This leads to the TSA and zero-tolerance policies that suspend six-year olds.

A better response is to stop treating people like interchangeable components in a machine and start designing with an eye toward integrating human limits and human creativity into our systems. Assume that the new system will produce unexpected results. Focus your design effort more on swinging the balance toward pleasant surprises and less on eliminating surprises altogether.

Management and messiness

Clay Shirky

Image via Wikipedia

[Cross posted at FASTforward blog]

I’ve been mulling over Clay Shirky‘s remarks yesterday at FASTforward09. The bookends to his talk hint at some key challenges to managers contemplating their entry into the world of social media and Enterprise 2.0. Clay’s opening five word summary of Enterprise 2.0 is simply “group action just got easier.” While he shared a number of excellent stories and lessons, it was his closing discussion of how Amazon added social elements to its existing pages that I want to focus on.

By Clay’s count there are some 16 different social elements that are today part of the typical product page on Amazon. Each of these elements became part of the page as the outcome of an individual experiment. Amazon’s approach is to make it easy, and organizationally safe, to run experiments quickly and cheaply. While there is a technological component to making this experimentation cost-effective, it is the management and cultural aspects that are critical to success.

What Clay is calling attention to is the value to be found in encouraging the fundamental messiness and disorder of invention and discovery. Unfortunately, managers generally don’t become managers because they are fond of disorder. Even managers who have long ago abandoned the caricatures of command and control models are likely to find guiding this kind of innovation a source of discomfort. But it is discomfort that is essential to encouraging the sort of retail level innovation made possible in the technology environment that is emerging.

Nobel Laureate Linus Pauling once observed that “the best way to have a good idea is to have lots of ideas.” That’s the mechanism at work at Amazon and with Enterprise 2.0 innovation in general. What Clay skipped over in his remarks was a look at the number of ideas that were tried and never made the cut at Amazon. This is unfortunate because it can encourage executives to ignore the “lots of ideas” prerequisite to “good ideas.” Amazon’s approach is sometimes portrayed as lowering the cost of failure. More appropriately, it is about lowering the costs of all experiments. While the technology environment is one factor in lowering the cost of experimenting, there are also managerial and cultural costs to manage. For example, if you insist on wrapping too much methodology and project management overhead around experimenting that will discourage ideas and fewer ideas implies fewer good ideas.

This is not a suggestion that there is nothing to manage. Instead, it’s about seeking just enough control. It’s also about becoming comfortable with trusting your people and the process of experimentation and learning.


Reblog this post [with Zemanta]

Simple competence as an overarching theme for 2009

I’m increasingly fond of Bob Sutton’s work at Stanford. Here’s a recent post of his on a critical observation from organizational theorist James March.

After working with Stanford’s bureaucracy for month and months to try to get a scholar appointed and paid (we have the money, that is not the problem), and still not having luck, I was reminded of a lovely and rather obscure article by James March   It was based on an address that he gave to academic administrators at the University of Illinois in 1980.  This excerpt seems especially appropriate at the moment:

“The importance of simple competence in the routines of organizational life is often overlooked when we sing the grand arias of management, but effective bureaucracies are rarely dramatic…. Much of what distinguishes a good bureaucracy from a bad one is how it accomplishes the trivia of day-to-day relations with clients and day-to-day problems in maintaining and operating its technology.  Accomplishing these trivia may involve considerable planning, complex coordination, and central direction, but is more commonly linked to the effectiveness of large numbers of people doing minor things competently. As a result, it is probably true that the conspicuous differences around the world in the quality of bureaucratic performance are due primarily to variance in the competence of the ordinary clerk, bureaucrat, and lower manager, and to the effectiveness of routine procedures for dealing with problems at a local level.  This appears to be true of armies, factories, postal services, hotels, and universities.”

Right now, some simple competence sounds pretty damn good to me. As you may have gathered, March is not much of a fan of heroic leaders, he believes more in well-designed systems filled with competent people.  His quote also reminds me of one I heard from the folks at the Institute for Health Improvement (they credited the Army Corp of Engineers): Strategy is for amateurs; execution is for professionals. 

The above quote is from “How We Talk and How We Act: Administrative Theory and Administrative Life,” which March published here. For a general tour of his work, check out Decisions and Organizations and The Pursuit of Organizational Intelligence.

P.S. The post is dedicated to that very patient scholar!

In Praise of Simple Competence
Sat, 27 Dec 2008 00:01:14 GMT

Helping individuals and organizations operate more competently seems to be an excellent underlying goal for 2009.

Reblog this post [with Zemanta]

Cisco as an emerging Enterprise 2.0 case example

Cisco Systems, Inc.

Image via Wikipedia

[cross-posted at FastForward blog]

The current issue of Fast Company has a cover article on Cisco and their ongoing efforts to reorganize into something that is an excellent case study of what Enterprise 2.0 may look like in an established organization. It shouldn’t be any surprise that the quintessential networking company is on the leading edge of network thinking applied to organizational design. At the same time, Cisco is a large, successful, hierarchical, engineering-centric organization that isn’t likely to be terribly interested in organizational fads.

Here’s the argument in a nutshell:

Chambers has greater ambitions, even now, in the midst of turmoil. Or, perhaps, especially now. He has been taking Cisco through a massive, radical, often bumpy reorganization. The goal is to spread the company’s leadership and decision making far wider than any big company has attempted before, to working groups that currently involve 500 executives. This move, Chambers says, reflects a new philosophy about how business can best work in a networked world. “In 2001, we were like most high-tech companies, with one or two primary products that were really important to us,” he explains. “All decisions came to the top 10 people in the company, and we drove things back down from there.” Today, a network of councils and boards empowered to launch new businesses, plus an evolving set of Web 2.0 gizmos — not to mention a new financial incentive system — encourage executives to work together like never before. Pull back the tent flaps and Cisco citizens are blogging, vlogging, and virtualizing, using social-networking tools that they’ve made themselves and that, in many cases, far exceed the capabilities of the commercially available wikis, YouTubes, and Facebooks created by the kids up the road in Palo Alto.

The bumpy part — and the eye-opener — is that the leaders of business units formerly competing for power and resources now share responsibility for one another’s success. What used to be “me” is now “we.” The goal is to get more products to market faster, and Chambers crows at the results. “The boards and councils have been able to innovate with tremendous speed. Fifteen minutes and one week to get a [business] plan that used to take six months!” As storm clouds form for the rest of the business community, he says, “We’re going to gain market share.” [“How Cisco’s CEO John Chambers is Turning the Tech Giant Socialist“, Ellen McGirt]

What makes this case study useful and interesting is its emphasis on organization not technology. There’s an undercurrent in the article that everything is all a bit “socialist” somehow and isn’t that a surprise, which I found annoying at points. The more interesting point is that a bunch of engineers and big-organization executives are essentially concluding that hierarchy isn’t scaling well enough to meet their goals.

More than anything else, this story provides a well-documented case study that is an existence proof to other skeptical executives that the combination of Enterprise 2.0 technologies and the right organizational principles and practices can succeed.

Reblog this post [with Zemanta]

Knowledge work and micro-processes

[cross-posted at Fast Forward blog]

Recently, I sat through a presentation about a Sharepoint-based intranet project to improve processes within the HR group of a medium-sized organization. The process in question was one of collecting annual performance reviews throughout the organization. Using Sharepoint, the HR group and their consultants replaced Word documents, spreadsheets, and email with Infopath forms and programmatic workflows. The client was happy and the consultants had a nice demo they could show to their prospects. Nonetheless, I found myself dissatisfied.

For all the new technology deployed, this effort struck me as an example of what my old friend and mentor Benn Konsynski calls "speeding up the mess." This HR process is an instance of the micro-processes that comprise knowledge work activities in organizations.

Other examples might include:

  • Customizing an existing sales presentation for a meeting with a new prospect
  • Designing the agenda and preparing materials for an internal brainstorming meeting
  • Putting together the briefing materials for a quarterly business review meeting
  • Analyzing and making sense out of a competitor

Useful models of systems change

All models are wrong. Some models are useful.
George E.P. Box

Although we’re constantly engaged in attempts to improve systems and organizations by introducing new practices and technologies, we still tend to do a mediocre job of dealing with the ensuing organizational changes. Part of our problem is that we tend to rely on very simplistic models of organizational change, when we think about it at all. I’ve recently been revisiting the issue in the literature and in my work. The organizing framework that I’m finding most robust and useful comes originally out of the work of Virginia Satir, a family therapist who adopted a dynamic systems perspective of interaction that translates nicely into organizational settings. I came to Satir’s work by way of Gerry Weinberg, a long time student of technology and systems induced change.

The following diagram captures the essence of Satir’s model of systems change:


In this model, the change process starts with the introduction of some “foreign element,” which might be a new system or a new manager or a new performance mandate from on high, disrupting the “Old Status Quo.” “Chaos” ensues and performance both falls and becomes more erratic until a “transforming idea” develops or is introduced. This transforming idea constitutes a new theory about how to operate in the new system. With a good transforming idea in place, there is a period of integration and practice as the organization learns how to perform in its new configuration. Eventually, we reach a “New Status Quo” operating at a new, hopefully higher, level of performance.

There are a number of features of this model that I find valuable. First, it acknowledges that performance is always variable, even during periods of relative stability. It’s an important reminder that the systems we are talking about are made up of people and we must allow for their humanity. Second, it makes it clear that change is fundamentally a learning process not a deployment process. Moreover, it is “learning how” not “learning that,” which should help us keep in mind that things will get worse as an inevitable and necessary part of getting better and that the learning will take time.

Finally, there is the notion of a “transforming idea.” All too often, by the time we get to deployment, we’ve forgotten why we embarked on the journey to begin with. Articulating and sharing an effective transforming idea is an essential step in achieving the new levels of performance we are seeking. Human systems are homeostatic; they seek out and maintain stability. Absent a compelling transforming idea, these homeostatic forces will drag the system back to its current status quo. With a good “big picture” in mind, the participants in the changing system have a goal that can guide them through the necessary integration and practice.

Cognitive surplus and organizational slack

[Cross posted at FASTforward]

Clay Shirky’s got a new talk and he’s taking it on the road. It’s stimulating a good bit of thoughtful discussion around the web. Here’s a video version of his talk:

Shirky has also posted a transcript of the talk on his site, if you’d prefer to read instead of watch. The talk is a riff on one of the themes of his new book, Here Comes Everybody: The Power of Organizing Without Organizations. I’ll post a complete review of that shortly; it’s well worth you’re making time to read it.

One of the stories Shirky hangs his argument on is an interchange with a TV producer about the creation and growth of Wikipedia. Here’s how he tells it:

I started telling her about the Wikipedia article on Pluto. You may remember that Pluto got kicked out of the planet club a couple of years ago, so all of a sudden there was all of this activity on Wikipedia. The talk pages light up, people are editing the article like mad, and the whole community is in an ruckus–“How should we characterize this change in Pluto’s status?” And a little bit at a time they move the article–fighting offstage all the while–from, “Pluto is the ninth planet,” to “Pluto is an odd-shaped rock with an odd-shaped orbit at the edge of the solar system.”

So I tell her all this stuff, and I think, “Okay, we’re going to have a conversation about authority or social construction or whatever.” That wasn’t her question. She heard this story and she shook her head and said, “Where do people find the time?” That was her question. And I just kind of snapped. And I said, “No one who works in TV gets to ask that question. You know where the time comes from. It comes from the cognitive surplus you’ve been masking for 50 years.”

So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project–every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in–that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it’s a back-of-the-envelope calculation, but it’s the right order of magnitude, about 100 million hours of thought. And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that’s 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, “Where do they find the time?” when they’re looking at things like Wikipedia don’t understand how tiny that entire project is, as a carve-out of this asset that’s finally being dragged into what Tim calls an architecture of participation. [Gin, Television, and Social Surplus]

The notion of “cognitive surplus” is a clever and useful way to frame the issue. Now, Shirky is primarily interested in the societal level impacts of new technologies. Big numbers help his argument tremendously, but they are a little bit like the arguments for why you might want to target your new consumer product at China (“if we only get one person in a hundred to drink our new sport drink, we’ll sell millions!”). Or the dotcom era arguments for capturing eyeballs. I don’t think that Shirky falls into this trap himself. Here, and in his book, he explicitly talks about how the design and architecture of systems such as Wikipedia leverage cognitive surplus in granular ways to exploit these large numbers.

My primary interests are inside organizations. How can we translate and adapt these insights into those environments? Organizational theorists, not being as clever or market oriented as Shirky, did not think up a notion as attractive as “cognitive surplus.” Instead, they talk about the notion of “organizational slack.” In hindsight, a very poor choice of words. For the last two decades, or more, organizations have been rooting out “slack” wherever they could find it. When the goal is efficiency, this is an appropriate strategy. However, it leaves no capacity for innovation and adaptation. Those few organizations that explicitly provide this capacity, such as Google’s 20% rule, are deemed notable and newsworthy.

The first order of business for business is to immediately appropriate Shirky’s term. Organizations that care about innovation and adaptive capacity should begin talking about “cognitive surplus.” Look for ways to measure it, if only crudely, and increase it.

The second task is to better understand and appreciate how various new technologies and tools let organizations derive benefit from smaller grains of cognitive surplus. Google’s 20% rule is a product of a time largely before blogs and wikis. Can an organization combine the tools with a one hour or 10 minute rule? Can we get value out of an hour a week or 10 minutes contributing to an internal wiki? Clearly, we will need to design some thoughtful support and encouragement processes around the tools in order to take advantage of a different scale of participation.

The third task is to monitor how well the large number phenomena outside the enterprise operate inside. We may discover critical mass issues; efforts below a certain scale are doomed to fail, while slightly larger efforts will need an extensive “life-support” system to survive. Other efforts may need support scaffolding but can become self-sustaining. Today, we have far more questions than answers. Shirky has provided us with some good new notions to start finding answers. I’d also recommend some of the following discussions that I’ve come across so far: