The False Wisdom of Crowds

If you had a choice of flying from Boston to San Diego in a plane piloted by a single machine or the combined intelligence of 3,000 people, which would you choose?

Perhaps you would want more information before making that decision.

If the machine piloting the aircraft was a well-designed piece of equipment that had been used as an autopilot for thousands of successful flights, and the 3,000 people were of average intelligence living in the Midwest, which would you choose?

What if the 3,000 people consisted of the combined intelligence of the best airline pilots in the aviation industry?

When working with large groups of people online, the wisdom of crowds is neither elevated to the smartest among them, nor is it diminished to the lowest levels. It hoovers somewhere in the middle.

But the cumulative influence of crowds, in today’s society, is very influential. And there is a pervasive notion that the crowd is always right. But what happens when it is wrong?

A History of Wrong-Headed Crowds

Leading up to 1929, it was a well-known fact that stocks were a great place to put your money.

In the 1950s, it became common knowledge that if a nuclear bomb went off in your city that you’d be safe if you simply learned to “duck and cover.”

In the 1970s, because of all the TV shows, virtually everyone knew about “quick sand” and how dangerous it was.

Up until 2007, it was a well-known fact that real estate was a great investment where you would virtually never lose money.

Today’s movies give most young people the idea that anyone can jump from a car going 30 mph, roll a few times, and be fine.

So much for the wisdom of crowds.

Drew Curtis, founder of Fark.com, has a very low regard for group intelligence.

“The ‘wisdom of the crowds’ is the most ridiculous statement I’ve heard in my life. Crowds are dumb,” Curtis says. “It takes people to move crowds in the right direction, crowds by themselves just stand around and mutter.”

Curtis points to his own experience moderating comments on Fark, where users present a rather humorous take on the news of the day. He says, “Only one percent of Web comments have any value and the rest are just garbage.”

Leveraging the Thinking of Crowds

When Congress wants to study an issue, they don’t bring in average people to testify. They only want the experts.

On the other end of the spectrum, it really doesn’t work to poll people on their thoughts about winning lotto numbers to find the winning combination.

However, polling crowds can be a very good tactic for determining a product’s success if the crowd is closely aligned with that product’s target market.

While people are very good at knowing what they personally want, they are generally very bad at understanding the truths of the world around us.

A study last year by ETH Zürich found social influence is a huge factor that can greatly undermine the “wisdom of crowds.”

They found that influential people have a way of narrowing the diversity of opinions to the point where it undermines the wisdom of crowd effect in three specific ways:

1. “Social influence” diminishes the range of opinions.
2. This “range reduction effect” tends to polarize people’s opinions making group feedback less reliable in guiding decision-makers.
3. Influencers tend to improve people’s confidence, but this so-called “confidence effect” will boost an individual’s confidence, while at the same time, decrease their accuracy.

Next Generation Super-Influencers

The Internet has made it far easier to gather information about people’s tendencies, inclinations, and opinions.

It has also made it far easier for super-influencers to polarize people’s judgement.

The whole democratic process in the U.S. today has really evolved into more of a battle of the super-influencers than the traditional gamesmanship by the candidates themselves. As a result we are seeing an increasingly polarized electorate.

Even though an elite group of super-influencers can be leveraged in the wrong way, marketing people are quick to get these same super-influencers to endorse their products and services in a more positive fashion.

Big Data’s New Wave of Decision-Making

In the past we have had the luxury of being able to take time making a decision.

Decision-making groups like planning boards, town councils, or some other overseer of a process, generally meet on a liesurely basis. Big decisions are often sliced into multiple smaller decisions and handled over longer periods of time. This time honored system for making decisions allows the general public to give their input and recommend changes along the way. In general, a fair way of doing business, but also a slow way of getting things done.

This slow determanistic model is being replaced with data mining techniques that can access the opinions of large constituencies or the buying whims of a target market almost instantly.

The speed of decision-making has become a competitive advantage in the business world and the latest form of bragging rights when it comes to communities. Yesterday’s taildraggingly slow processes are being replaced with an entirely new social norm.

According to a recent study by Capgemini, big data is becoming as fundamental to business as land, labor and capital.

“It’s not only through harnessing the many new sources of data that organisations can obtain competitive advantage. It’s the ability to quickly and efficiently analyze that data to optimize processes and decision making in real time, that adds the greatest value,” says Capgemini Global Sales Director Paul Nannetti. “In this way, genuinely data-driven companies are able to monitor customer behaviors and market conditions with greater certainty, and react with speed and effectiveness to differentiate from competition.”

Gems of real human intelligence will be captures as big data begins to comprehend larger trends and structures.

Capturing Real “Human” Intelligence

A few years ago I was involved in a search engine research project where we looked closely at the connecting strategy people used to analyse the relationship between a search phrase and the resulting website that they were eventually looking for.

Once users typed in a search phrase, we studied how the discernment process unfolded, with inappropriate sites being discarded before a final destination was chosen. Over time it became clear that the search path itself was layered with huge amounts of valuable data that should be captured and dissected for later use.

The information fragments that we were capturing were not merely data points along a line; we were capturing actual pieces of real human intelligence. Since real people were making the link between the search terms and the destination site, albeit a primitive association, it was indeed a useful nugget of human thinking.

Over time, artificial intelligence will give way to far more useful forms of real human intelligence.

Perhaps a better way to explain the capturing of real human intelligence is to explain my vision for the music player of the future. Since music is a very integral part of our lives, we can all relate to the power of listening to the right song at just the right time. But, for each of us, that perfect “right song” is always different.

So let’s imagine a music player that only played the “right songs”. One great song followed by another.

The ultimate music player will be capable of doing exactly this. It will assess our mood, our likes and dislikes, understand the contextual attributes of time, place, and people, decide whether we’re doing something that requires us to be physically active or just sitting comfortably in a chair, and it will anticipate our response to the music before it is ever played.

The ultimate music player will measure our heartbeat, brainwaves, biorhythms, stress levels, circadian rhythms, and a few other sensory inputs we haven’t even invented yet.

This future music player will only serve up songs that our body has a positive response to.

Technology like this will elevate our minds, and mental clarity, to a whole new level. It will be simultaneously energizing and relaxing, and give us motivation, endurance, and determination. In short, it will give us a reason to bound out of bed every morning to tackle a shining new day.

A device like this will require capturing real intelligence in real time.

As we move into the big data era, we will continue to uncover far more sophisticated ways of both capturing and leveraging these gems of real human intelligence, and finding unusual ways of using only those with the highest grade of intelligence behind them.

Final Thoughts

Today’s so-called “wisdom of crowds” will be replaced by three driving forces:

1. Next Generation Super-Influencers
2. Big Data’s New Wave Decision-Making
3. Real Human Intelligence

While it will still be important to keep people engaged in a participatory fashion, the value of these three forces will quickly supercede the complexity of crowds.

When it comes to big data, IBM estimates that 2.5 quintillion bytes of data are created every day. Using this metric means that 90% of all the data in the world today has been created in the last two years.

Out of big data will come big strategy which will begin with effectively framing the problem.

A recent study by McKinsey and Company calculated an immediate shortage in the U.S. of 140,000-190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis.

The biggest gap is the lack of geek-savvy managers capable of making good decisions in this type of environment.

Over the coming months, the people-friendly “wisdom of crowds” is about to be railroaded by the oncoming big data train. To many it was a passing fad, to others, it never stood a chance.

About the author:
Thomas Frey is the innovation editor for THE FUTURIST magazine and author of the book Communicating with the Future. This article was originally posted on his Website Futuristspeaker.com