Beating the Hive Mind

“What’s this?” I asked, toying with a white cylinder with letters printed across it.

“It’s a cryptex,” explained Eric Harshbarger, one of Mind Candy’s in-house puzzle designers. “Like the one from the Da Vinci Code.”

In The Da Vinci Code, a cryptex is a cylinder with five wheels that can be rotated independently; each wheel has letters printed on it, and if you line up the wheels properly, it’ll open. The puzzle to open the cryptex in the movie was rather boring, but Eric had come up with a much more interesting multi-stage puzzle and then constructed it himself. He’d brought the cryptex, along with some other fun physical puzzles, to San Francisco for our live event there last year.

While we walked down to a nearby cafe for breakfast, Eric mentioned how he’d visited Google a couple of days earlier with the cryptex and shown it to some of the puzzle-fans there, including Wei-Hwa Huang, the designer of Google’s Da Vinci Code puzzle quest. Immediately, Wei-Hwa and two other Googlers threw themselves on the task, and within two or three hours, had figured it out. Thus the challenge was set: could we beat Google?

Personally, I didn’t think so. Those guys not only live and breathe puzzles, they actually spend a lot of time solving them. So I passed (I didn’t have a few hours to spare), and instead played around with some of Eric’s wooden puzzles while David Varela, a writer at Mind Candy, busied himself with the cryptex.

Thirty minutes later, David had solved the cryptex. He had beaten Google. And he didn’t even have a computer, let alone a piece of paper.

As we slowly descended from that glorious, legendary moment, we reflected on how this could be possible. David’s no slouch in the puzzle department, but to beat three of Google’s top puzzlers, unaided, was really quite amazing. So what had happened? Thankfully, we had an eyewitness account from Eric about the Google team’s strategy, since he’d watched them attack the cryptex.

When they received the cryptex, they immediately began swapping theories on the possible ciphers it used. Was it a Caesar cipher? Simple substitution? Vigenere? Someone immediately began transcribing the cryptex onto their computer. People coded little programs to look for solutions. As the minutes and hours went by, more theories were advanced and retracted, and several dead-ends were explored at length. Eventually though, they backtracked and made it through.

It seemed that the reason they took longer than David was precisely because they were three people, and he was only one. Three people can generate many more theories than a single person, and so – except for obvious or easy puzzles – they have a higher chance of coming across the correct theory. However, each theory demands at least some explanation and consideration, which takes time. More damagingly though, having a group attack a puzzle introduces the dangers of groupthink: someone will inevitably suggest a correct theory, only to have it either ignored or dismissed out of hand by the other group members. Eric noted that this happened a few times with the Google team. Only later, after losing time in dead-ends, would they return to the old theory.

In contrast, an individual stands less chance of being distracted – after all, they only have themselves to talk to when working on puzzles. And this is why an individual was able to solve the cryptex significantly faster than a group – the interactions between the group members created friction that slowed down the attack.

Of course, what I’m telling you here isn’t the full story. I haven’t seen a single person solve the cryptex as fast as David did. To my knowledge, no indivudal besides David has solved it as fast as the team at Google, so this tells us something else: while an individual may be able to beat a group, a group will almost always beat an individual. Even though the Google team was distracted and slowed down by wrong theories, they eventually got back on the right track because they had a greater chance of finding the correct theory among all of those they generated; they were able to generate and explore possibilities faster.

I brought this up with Wei-Hwa when I met him in San Francisco a couple of weeks ago. He took this idea further, and asserted that no group could solve a puzzle like Eric’s cryptex in ten minutes (or thirty minutes). Only an individual would have the focus and lack of distractions to solve it that quickly. But on average, groups would solve the cryptex much quicker. I suggested a graph that could describe this:

This wholly unscientific graph (ignore the axes) shows an individual having an advantage due to lack of distraction for, say, the first hour, after which the increased brainpower of the group begins to outgun the individual. I don’t think that it’s impossible that a group couldn’t solve the cryptex in thirty minutes – I just see it as being highly unlikely. To do so, they’d basically have to stop talking to each other, which would remove the more powerful effects of group co-operation that kick in later.

I had an opportunity to see these behaviours first-hand during Perplex City Season 1. Whenever people were working on a difficult and complex puzzle, the same problems that Eric described would happen – people would advance correct theories that were unaccountably ignored and dismissed for days if not weeks. This problem, if unchecked, is exacerbated on the internet where people who aren’t willing or confident enough to defend their theories will simply accept their dismissal, and others who are watching will simply assume that a theory people aren’t working on is clearly not worth pursuing.

Of course, when someone had demonstrated that a particular theory was correct*, everyone following the discussion now had the solution, or at least immediately advanced to the next stage of the puzzle, thus eliminating any wasted effort. And of course, the number of people working actively working on a puzzle – dozens or hundreds – increased the chance that the correct theory would eventually be found. If groups could change its behaviour to properly evaluate all theories put forward, I think there would be a significant gain in performance.

However, I believe I have shown that there are some problems that a ‘hive mind’ cannot solve as quickly as an individual. Following from that, I also believe that the hive mind is inimical to certain types of very difficult or complex scientific and mathematical problems – such as quantum physics or the Poincare Conjecture**. Without revolutionary gains in communication (including communication technology) and co-ordination, I don’t think this will change.

Notes:

*: This leads to a useful lesson for puzzle design. If you’re creating a multi-stage puzzle, you should ensure that when players arrive at the solution for a particular stage, they should not have any doubt that they are correct, otherwise they may waste time exploring other possibilities.

**: I do think that there are some hard scientific problems that hive minds could excel at, especially those involving large amounts of data collection such as epidemiology. It remains to be seen whether these problems can be made sufficiently fun to tackle, or tractable by amateurs.