Mssv

Mssv random header image

Beating the Hive Mind

March 19th, 2007 · 3 Comments

“What’s this?” I asked, toying with a white cylinder with letters printed across it.

“It’s a cryptex,” explained Eric Harshbarger, one of Mind Candy’s in-house puzzle designers. “Like the one from the Da Vinci Code.”

In The Da Vinci Code, a cryptex is a cylinder with five wheels that can be rotated independently; each wheel has letters printed on it, and if you line up the wheels properly, it’ll open. The puzzle to open the cryptex in the movie was rather boring, but Eric had come up with a much more interesting multi-stage puzzle and then constructed it himself. He’d brought the cryptex, along with some other fun physical puzzles, to San Francisco for our live event there last year.

While we walked down to a nearby cafe for breakfast, Eric mentioned how he’d visited Google a couple of days earlier with the cryptex and shown it to some of the puzzle-fans there, including Wei-Hwa Huang, the designer of Google’s Da Vinci Code puzzle quest. Immediately, Wei-Hwa and two other Googlers threw themselves on the task, and within two or three hours, had figured it out. Thus the challenge was set: could we beat Google?

Personally, I didn’t think so. Those guys not only live and breathe puzzles, they actually spend a lot of time solving them. So I passed (I didn’t have a few hours to spare), and instead played around with some of Eric’s wooden puzzles while David Varela, a writer at Mind Candy, busied himself with the cryptex.

Thirty minutes later, David had solved the cryptex. He had beaten Google. And he didn’t even have a computer, let alone a piece of paper.

As we slowly descended from that glorious, legendary moment, we reflected on how this could be possible. David’s no slouch in the puzzle department, but to beat three of Google’s top puzzlers, unaided, was really quite amazing. So what had happened? Thankfully, we had an eyewitness account from Eric about the Google team’s strategy, since he’d watched them attack the cryptex.

When they received the cryptex, they immediately began swapping theories on the possible ciphers it used. Was it a Caesar cipher? Simple substitution? Vigenere? Someone immediately began transcribing the cryptex onto their computer. People coded little programs to look for solutions. As the minutes and hours went by, more theories were advanced and retracted, and several dead-ends were explored at length. Eventually though, they backtracked and made it through.

It seemed that the reason they took longer than David was precisely because they were three people, and he was only one. Three people can generate many more theories than a single person, and so – except for obvious or easy puzzles – they have a higher chance of coming across the correct theory. However, each theory demands at least some explanation and consideration, which takes time. More damagingly though, having a group attack a puzzle introduces the dangers of groupthink: someone will inevitably suggest a correct theory, only to have it either ignored or dismissed out of hand by the other group members. Eric noted that this happened a few times with the Google team. Only later, after losing time in dead-ends, would they return to the old theory.

In contrast, an individual stands less chance of being distracted – after all, they only have themselves to talk to when working on puzzles. And this is why an individual was able to solve the cryptex significantly faster than a group – the interactions between the group members created friction that slowed down the attack.

Of course, what I’m telling you here isn’t the full story. I haven’t seen a single person solve the cryptex as fast as David did. To my knowledge, no indivudal besides David has solved it as fast as the team at Google, so this tells us something else: while an individual may be able to beat a group, a group will almost always beat an individual. Even though the Google team was distracted and slowed down by wrong theories, they eventually got back on the right track because they had a greater chance of finding the correct theory among all of those they generated; they were able to generate and explore possibilities faster.

I brought this up with Wei-Hwa when I met him in San Francisco a couple of weeks ago. He took this idea further, and asserted that no group could solve a puzzle like Eric’s cryptex in ten minutes (or thirty minutes). Only an individual would have the focus and lack of distractions to solve it that quickly. But on average, groups would solve the cryptex much quicker. I suggested a graph that could describe this:

This wholly unscientific graph (ignore the axes) shows an individual having an advantage due to lack of distraction for, say, the first hour, after which the increased brainpower of the group begins to outgun the individual. I don’t think that it’s impossible that a group couldn’t solve the cryptex in thirty minutes – I just see it as being highly unlikely. To do so, they’d basically have to stop talking to each other, which would remove the more powerful effects of group co-operation that kick in later.

I had an opportunity to see these behaviours first-hand during Perplex City Season 1. Whenever people were working on a difficult and complex puzzle, the same problems that Eric described would happen – people would advance correct theories that were unaccountably ignored and dismissed for days if not weeks. This problem, if unchecked, is exacerbated on the internet where people who aren’t willing or confident enough to defend their theories will simply accept their dismissal, and others who are watching will simply assume that a theory people aren’t working on is clearly not worth pursuing.

Of course, when someone had demonstrated that a particular theory was correct*, everyone following the discussion now had the solution, or at least immediately advanced to the next stage of the puzzle, thus eliminating any wasted effort. And of course, the number of people working actively working on a puzzle – dozens or hundreds – increased the chance that the correct theory would eventually be found. If groups could change its behaviour to properly evaluate all theories put forward, I think there would be a significant gain in performance.

However, I believe I have shown that there are some problems that a ‘hive mind’ cannot solve as quickly as an individual. Following from that, I also believe that the hive mind is inimical to certain types of very difficult or complex scientific and mathematical problems – such as quantum physics or the Poincare Conjecture**. Without revolutionary gains in communication (including communication technology) and co-ordination, I don’t think this will change.

Notes:

*: This leads to a useful lesson for puzzle design. If you’re creating a multi-stage puzzle, you should ensure that when players arrive at the solution for a particular stage, they should not have any doubt that they are correct, otherwise they may waste time exploring other possibilities.

**: I do think that there are some hard scientific problems that hive minds could excel at, especially those involving large amounts of data collection such as epidemiology. It remains to be seen whether these problems can be made sufficiently fun to tackle, or tractable by amateurs.

Tags: arg · science · spec

3 responses so far ↓

  • Brian Enigma // Mar 19, 2007 at 3:52 am

    It seemed that the reason they took longer than David was precisely because they were three people, and he was only one. Three people can generate many more theories than a single person, and so – except for obvious or easy puzzles – they have a higher chance of coming across the correct theory. However, each theory demands at least some explanation and consideration, which takes time.

    This sounds not unlike The Mythical Man Month. If you’re not already familiar with the book, the theory goes something like this: as you add people to a problem that needs solving, you introduce more communications/management overhead, causing everything to take longer. Eventually you reach a threshold where people are doing more communications and task switching than actual work. That’s why a task requiring 40 man-hours can be completed by an individual in a week, but (with few exceptions) can’t be done by 40 people in an hour.

    People would advance correct theories that were unaccountably ignored and dismissed for days if not weeks. This problem, if unchecked, is exacerbated on the internet where people who aren’t willing or confident enough to defend their theories will simply accept their dismissal, and others who are watching will simply assume that a theory people aren’t working on is clearly not worth pursuing.

    Also, people would assume that something that was dismissed was already worked on by someone else. As an individual it’s much more fun and glamorous to come up with your own theory and prove/disprove it than it is to take a bunch of proposed theories that may or may not have been tested by others and double-check them yourself.

  • Margaret // Mar 19, 2007 at 10:03 am

    Absolutely fascinating post. Sounds like you’ve just formulated ‘Adrian’s Law of Group Performance’! I think the case of group work in Perplex City also falls into rather a different category from the type of group work in the Google example, one that involves a major element of individual work as well, somewhere in between the two on the graph. You’re right that people discuss and often wrongly dismiss theories, but there are also a lot of people individually working on the puzzle. There have been many occasions where a correct result is achieved not because the group did it together, but simply because there is such a large pool of players to draw upon. With thousands of people trying to crack a puzzle, one of them is bound to get it eventually. This is even more true when the puzzle requires a highly advanced skill. It might be something that only a few people would have the specialist knowledge to crack, but when you have a group of thousands, you statistically increase your chances that *someone* will know how to tackle it. Also, communication between the PXC group through forum posts and IRC certainly differs from people working together in person, probably slowing it down even more and with the linear format of thread-posting increasing the potential for the correct theory to be lost amongst the chatter. With the size of group too, there’s also always a number of people diluting the discussion with random comments who aren’t even participating in the solve attempt. So the PXC group does manage to incorporate some of the benefits of individual work, but the format of communication and the group itself can hamper efforts. Still, all-in-all, a pretty extraordinary group! Thanks again for the food-for-thought.

  • David Varela // Apr 15, 2007 at 12:27 am

    Adrian – I’ve only just found this article. Bless you, bless you for immortalising my finest (half-)hour.

    I suspect that context may play a significant part in how any group or individual approaches a puzzle. If you were offered a wooden puzzle over breakfast, you wouldn’t expect it to use military-grade encryption. However, if you’re a crack puzzle team from Google and you’re being challenged by one of the world’s great puzzle brains (that’s you, Eric), your expectations are markedly different. You automatically assume the answer must be obscure. You dismiss the obvious.

    So, because Eric gave the puzzle to *me*, I already knew it couldn’t be that tough. I not only had the advantage of being one person, but also of knowing that the solution must be simple. Eric just isn’t that cruel.

    Also, my relative ignorance reduced the number of theories I could come up with. That accelerated my solve, but it also increased the chances of me being totally stumped. The solution happened to fall within my narrow area of knowledge.

    I’d therefore suggest an amendment to your graph. A group is likely to get there in the end, whereas an individual is much more likely to be unequipped for the task. After a certain point – earlier than that for a group – the chance of a solve for an individual will decrease sharply. If you don’t get it quick, you probably won’t get it at all.

    That’s before we even take into account the social pressures of giving up on a task when you’re part of a group. I’d suspect that a group would generally persist longer than an individual, but now I’m heading well beyond my narrow area of knowledge again…

Leave a Comment