Are we overvaluing real-time feedback?

Posted on February 6, 2010 4:38 pm by | Business | Channels | Crisis Communication | Marketing | PR | Presentations | Social Media

Warning: Lost post follows

imageBack in 1995, “Snow Crash” author Neal Stephenson teamed up with his uncle George Jewsbury under the pseudonym Stephen Bury to produce a potboiler titled “Interface.” The premise: A presidential candidate suffers a stroke and has a chip implanted in his brain. The chip features a wireless connection to feedback from thousands of watch-like devices distributed to a representative sample of Americans. These devices gauge the wearer’s reaction to political speeches, allowing the candidate to make mid-course adjustments and bolster public reaction to his candidacy.

To me, this bit of speculative fiction defines the notion of a real-time feedback loop.

As the Web proceeds along its evolution into a more real-time network, a idea of a real-time feedback loop is becoming a popular topic of discussion. I attended a panel discussion on Thursday night, part of Social Media Week here in San Francisco, that focused on these loops, defining them as “a method for capturing ideas as they arise and bringing them back into the group for examination through the use of social media.” Promotional copy for the event asserted:

When an idea???s expression generates a creatively relevant or insightful response, a well-organized listening/engagement practitioner captures that flash of brilliance, and feeds it back to the originator as an enriched question, thus creating a real-time feedback loop.  In this transformational moment, a thought-leader may have a second opportunity to be heard and have their expression innovatively re-cast. 

With social media we facilitate this process ever more effectively. It is like cold fusion???when used properly, it creates more value than it consumes, lowering the carbon footprint of innovation.

The idea of real-time feedback loops have been rattling around in my brain since Thursday night’s discussion. Then it occurred to me: What better place to organize my thoughts than my blog?

Where do real-time feedback loops begin?

The Internet didn’t invent real-time feedback loops. The thunderous applause of an audience that leads to a multiple curtain calls is a real-time feedback loop; so is tepid applause followed by a rush for the exits. The Grateful Dead’s symbiotic relationship with its audience influenced the band’s live improvisational music. The crowd’s response almost always affects a standup comic’s routine.

Shel Holtz

The Net, however, has added two dimensions to real-time feedback loops: specificity and reach.

Specificity—The aggregate response of the crowd is pretty simple. They love it, they’re into it, they disagree, they don’t think it’s funny, they hate it. The Net has provided individuals a voice that allow the performer or communicator to analyze why the crowd is reacting the way it is and respond to specific observations or alter behaviors in order to influence opinions. This is nothing new: For at least a decade, probably longer, Internet Relay Chat (IRC) has provided the infrastructure for backchannels, on which conference attendees discuss presentations with one another in real time. In some instances, these backchannels have been projected on a screen where a speaker can see and react to it. Now, Twitter’s hashtag convention—along with some other tools—have made backchannels available to more people than just the geek crowd who knew how to tap into IRC.

Reach—Streaming media and Twitter have expanded the reach of events—from keynotes and panel discussions to product launches and press conferences—to people who can’t be there in person. Again, this is nothing new. The presidential State of the Union address is one example of a speech that is available to larger audiences than just those who can squeeze into the chamber of the House of Representatives. The Net’s streaming capabilities, though, have made it possible to extend this ability to speakers and events that don’t warrant mainstream television network coverage. The most recent LeWeb, for example, was streamed to an audience hungry for presentations they couldn’t see in person due to the event’s cost (expensive) and location (Paris).

Combine these factors and the significance of real-time feedback loops becomes clear. Not only can an executive speaking at a product launch hear specific feedback in real time, but the audience is now expanded to customers or stakeholders from anywhere in the world.

Generally, this feedback comes in two forms: the general chatter of individuals expressing their opinions or talking with one another and targeted questions from individuals to the speaker. Both were in play last Thursday night as people watching a live stream of the presentation (courtesy of talked among themselves and posed questions for panelists that were relayed by an in-person moderator.

Shel Holtz

All eyes on real-time

It’s clear that the Net has altered and expanded realt-time feedback loops. Google has incorporated real-time results into its search results. A new category of real-time search engines has emerged sporting names such as Collecta, Topsy and Scoopler.

Prominent people are writing about the real-time web, including the authors of influential outlets like ReadWrite Web, GigaOn, Mashable and TechCrunch. Jeff Pulver, Stowe Boyd and Jeremiah Owyang have written about it. It found its way onto many 2010 prediction lists.

Protocols are being developed to support it. RealTime RSS—from RSS godfather Dave Winer—sends updates when they’re added to a site rather than waiting for an RSS reader or other utility to poll feeds to find what’s new. Google’s PubSubHubbub is similar although not necewssarily a competing standard; the two can work together. Chris Messina described PubSubHubbub’s function this way: “Let’s say (you write) a new blog post; the blogging software then pings any number of hubs with a message: ‘Hey, new content here.’ The hub says, ‘Great thanks,’ grabs the content, and then pushes the content to everyone on its ‘subscriber” list.’

These two protocols expand the opportunity for anyone to get real-time feedback. A marketing executive introducing a new product to a live audience and a virtual one watching the stream can hear back instantly from those engaged over conversational channels (Twitter and IRC, for example) as well as those writing for online news outlets and blogs.

As a result, the focus on real-time feedback has become intense. Some have proclaimed the ability to assess sentiment through real-time search a replacement for costly polling that has been the province of organizations like Harris and Gallup.

But how important is all this real-time feedback?

Is it accurate?

What you think at the instant you hear something may not be what you think after you’ve had time to digest it. Consequently, your immediate feedback may not reflect your long-term view.

This is one of the issues many speakers have with members of the audience live-tweeting their talks or with journalists live-tweeting events.

Much of the tweeting of live events is objective, though, rather than subjective. It’s more like note-taking than analysis. And even the opinions tweeted in real time have value. After all, you’re presenting in real time and people are reacting. Before, you could only see them shifting uncomfortably in their seats, or maybe actively booing or walking out. Now you can assess exactly why they’re reacting the way they are.

But in some respects, the critics have a point. Consider the widely-covered Apple iPad announcement. Information from Steve Jobs’ presentation was made available in real time through a number of channels and a lot (though certainly not all) of the real-time feedback suggested Apple had another sure-fire hit on its hands. But then came the analysis. Tech journalists, bloggers and others began producing the more thoughtful, detailed reviews after they had a chance to internalize the information, consider it, chew on it. FOr many members of the audience, digesting these views, then sharing them and discussing them with each other, led to a shift in their opinions. In the end, their early tweets didn’t reflect their ultimate views.

Is it representative?

During Thursday night’s panel, the point was made repeatedly that only about 10 percent of your audience will offer real-time feedback. And your larger audience—the customers for the product you’re launching, for instance—won’t even watch the event.

Reacting to real-time feedback, then, could mean that you’re taking action on information that isn’t representative of your customer base. In fact, those who pay attention to the live stream or real-time tweets of your message could be as far from a statistically valid sample of your population as you can get.

Is it contextual?

As I sat in the room where the panel was presented on Thursday, I was able to take in everything at once. There was the reaction of other panelists to what one panelist was saying, panel moderator Jennifer Lindsay‘s reaction, the panel’s reaction to Lindsay’s questions and the reaqction of the audience.

Shel Holtz

Those watching the stream, on the other hand, saw only what the camera allowed, and the camera was almost always focused on whoever was speaking. Those watching the stream got only a sliver of the experience had by those in attendance. IT’s even worse with those who see only the 140 characters broadcast by those who are live-tweeting the event. The reactions of those receiving these messages, then, could be based on incomplete or out-of-context information. It could conflict with the opinions of the people whose opinions you’re really trying to understand.

Because of these realities, the rush to embrace the real-time web can easily lead us to overvalue real-time feedback and make inappropriate decisions based on it.

When real-time feedback matters

Of course, recognizing the limits of real-time feedback doesn’t mean you shouldn’t be paying attention to it, only that you should be cirumspect in terms of what you do with it.

In a crisis, for example, you’d be foolish to ignore commentary emerging in real time. By monitoring public sentiment, you can determine the depth of reaction to the situation and quickly develop a response strategy. Real-time feedback in response to change initiatives is equally important. People resist change for a variety of reasons and listening to feedback can help you shape your efforts to overcome that resistance.

As for other feedback—to speeches, to announcements, to events—organizations will have to develop processes to determine which feedback requires immediate internalization and action and which becomes just additional information to factor into longer-term thinking. After all, how much can you really do with real-time feedback? We have no brain-implantable chips to help us adjust our comments in real time based on listener feedback. We can’t alter the presentation in mid-course when CNN’s cameras are on you. You can’t redesign the product if it’s already on trucks heading to retail stores. In most instances, real-time feedback won’t be more important than other forms of input, including the articles, reviews, blog posts, tweets and other consumer-generated content that will trickle out over days, weeks and months in response to your company’s message. Your best bet will be to add it to the mix in order to figure out your next steps, whether it’s a version 2.0 of your product, an enhancement to a program or a response to a query or criticism.

None of which means that engaging people through social channels is less important than it was before the real-time web became a hot topic. Engaging individuals through social channels isn’t necessarily the same as participating in a real-time feedback loop. Engaging in conversations, responding to questions and participating in communities is all part of an effort to establish strong relationships that will pay off over the long term.

Nor does this suggest that the real-time web isn’t important. The instant delivery of news means organizations have less time to prepare and more information through which to sift.

But when it comes to taking immediate action on the instant feedback to your message, tread with care. You could be solving a problem that doesn’t really exist.

Related post from Tom Foremski, who was on the panel (and is in the photo above): The Real-Time Web Turns ‘Conversational’ Media Into Noise

02/06/10 | 2 Comments | Are we overvaluing real-time feedback?



  • 1.Thanks for raising this conversation, Shel. It's long overdue and gets really interesting out at the edges.

    Remember that old saying "If you don't like the weather here, wait five minutes?" I think that's how it is with the real-time web. Immediate feedback paints one picture, but lengthen your feedback window and the picture can change dramatically.

    I've been taking a more conservative approach to teaching clients about real-time feedback lately. There's no questioning the real-time web's usefulness for finding answers to questions, catching a trend on the rise, etc.

    But most business decisions take far longer to make (for once, that MIGHT be a good thing) and have longer-term consequences. Making business decisions on the basis of real-time response can have disastrous consequences.

    Need I even mention Bush vs. Gore? Our real-time feedback on that was...well, less than accurate.

    Of course, human beings have been course-correcting based on immediate feedback forever. Immediate reaction vs. analysis are always in a state of tension.

    The question in my mind is whether we're significantly better off if one side or the other has the upper hand.

    Scott Hepburn | February 2010 | Charlotte, NC

  • 2.Shel,

    Interesting post, especially after I read the recent Mashable article that showed the initial negative sentiment (which is still being mentioned by the media) over the iPad dropped significantly after the initial product launch. Since, the iPad positive sentiment has grown to more than 70 percent.

    Had Apple responded in real time (not that they would have), they might have aggravated the situation with the lifespan of thirty minutes.

    All my best,

    Richard Becker | February 2010 | Las Vegas

Comment Form
What is the four-letter acronym for Bring Your Own Device?

« Back