The Mapping Problem

Issue 2 of my newsletter – subscribe here

There’s the book I’d like to write. And there’s the book I ought to write.

Last year, someone from Samsung’s strategy team asked to talk to me about whether “gamification” could be a possible use (if not a killer app) for a hypothetical augmented reality heads-up display.

I admitted, yes, Zombies, Run! was an example of gamification in action, although I quickly explained that I detested the term. Gamification “experts” will tell you that merely applying a coating of points, achievements, levels, and progress bars to otherwise-boring activities is not the way to turn it into a fun or engaging game – but promptly ignore their own advice when designing games for big companies that want to improve productivity, games that invariably are only about points andz achievements and leaderboards. So no, I didn’t think that gamification – as commonly understood and practiced – was a killer app for anything.

“But isn’t Zombies, Run! popular? Couldn’t you do that for other things?”

I sighed inwardly, wondering if I would save all the time I spent talking about why gamification sucked by writing an article or book instead. I wasn’t that simple, I explained, because of the Mapping Problem.

The Mapping Problem came to me at a games conference when I was asked if I could make mopping the floor more fun. Now, I don’t mind doing chores, but mopping is hard. It makes a mess, it requires more effort than vacuuming, and you don’t have the satisfaction of seeing bits of dirt disappear. It’s also something you know you ought to do more often, so it’s really a very good candidate for gamification because you’re intrinsically motivated to get it done.

The standard playbook of gamifying mopping would be an app that gave you points for each time you mopped the floor, and to give extra points and badges if you mopped regularly. Maybe you’d even give more points for mopping different parts of the house!

But this is a shit game design. Getting points and badges after your first session of mopping might make you feel slightly better, but after the third or fourth or eighth or ninth time you tap the “I’ve mopped, yo!” button, you realise that it’s meaningless. You’re just doing this for points. Meaningless points. The app doesn’t really care whether whether you did a good job; for all it knows, you’re just jabbing the button without doing any mopping at all.

So here’s what I would do. I would try to make the act of mopping more fun. I don’t want to reward you after you’ve finished mopping, I want you to be excited while you’re mopping. Ideally, I want you to be excited to start mopping because you know it’s going to be fun. This cannot be achieved through points and badges alone. It requires real-time motivation, either through gameplay or storytelling, or both.

I’m not sure storytelling quite fits mopping, so I’ll go with gameplay. And what I’m looking for is a kind of gameplay that maps onto mopping in a natural way, in a way that will help you achieve your ultimate goal: a thoroughly-cleaned floor. This will require the game to ‘know’ the cleanliness of the floor, which is probably most easily done with computer vision. Likewise, we need a feedback mechanism so the game can respond to what you’re doing, ideally using some flavour of augmented reality, as you are manipulating the real world. And this is the point where we get into the limitations and opportunities of technology, in the sense that this game isn’t possible unless a large number of potential players have either:

a) Multiple static home cameras that can see a majority of your floor

b) Body or head-mounted cameras

c) Periodically waving your phone’s camera at the floor No! This is a terrible idea.

Basically we need to wait for an affordable augmented reality heads-up display, which is a few years out. I honestly don’t think it’s possible to make an effective mopping game without this technology, any more than it’d be possible to make Zombies, Run! without GPS-enabled smartphones. Without it, the game has no idea whether you have mopped a particular part of the floor. Maybe you can use accelerometers to sense a back-and-forth motion (yes, haha) but that doesn’t tell you about the position of the mop. So cameras it must be.

It’s unusual for technology to so overtly be the limiting factor in game design. Most designers sensibly, and unconsciously, filter out game designs that are plainly not possible with current technology. Sure, you might have games that push graphical or processing boundaries, but few games push beyond existing interface technologies. Whenever they do – like Nintendo Labo or Guitar Hero or Pokémon Go or Beat Saber – we all get tremendously excited.

I’d argue that any gamification that extends to the physical world (as opposed to entirely digital activities like, say, gamifying online language learning) falls into this category where practically every solution requires some kind of custom interface technology. That’s just one of the reasons why gamification is hard. If you’re creating the next Fortnite or Mario or Grand Theft Auto, you don’t need to invent or harness an entirely new interface; but if you want to make the best game to teach you the violin, you have some very basic interface questions to answer.

(Incidentally, this is why touchscreens are so central to the success of smartphones: the technology is so versatile, it can solve entire swathes of interface problems. The same screen used for a first-person shooter like Fortnite can also be used for a match-three puzzler like Candy Crush. And that’s why effective AR heads-up displays have the potential to solve an order of magnitude more interface problems: you can use the technology for helping people to mop, or translating street signs, or making learning the violin more fun. So the Samsung guy wasn’t wrong, per se.)

OK, enough about technology. What’s the gameplay for our mopping game? Take your pick! Maybe it’s wiping away bugs crawling around the floor; or it’s a colouring game; or a game of Snake; or you’re sculpting a 3D landscape. There are lots of possibilities, the important thing is for the game to organically encourage you to cover the entire floor in a way that isn’t too rote or tiring. The point is, the game doesn’t just reward you for having mopped, rather, the act of mopping is the game.

But here’s the sting in the tail: the Mapping Problem means that even if we’ve solved the Mopping Problem, we haven’t solved the Violin Problem, or the Ironing Problem, or the Bin-Emptying Problem. These are all completely different kinds of activities that need to be made entertaining in different ways. So my objection to “gamification” is not that I hate the idea of making ironing more fun. I would love for such a game to exist. It’s that the term has come to mean the application not merely of a standard process, but a standard set of game mechanics, to wildly different problems – and any so-called “gamification expert” who claims otherwise is being wilfully blind to the reality of the shitty industry they have helped build.

Unsurprisingly, I have a whole lot more to say about gamification. An entire book outline and proposal, in fact. It’s a book I’m uniquely qualified to write, with a background in experimental psychology and neuroscience, plus the badge of creating one of the most successful examples of gamification ever. I even know it would sell decently well, since gamification never seems to die (“What is dead…” indeed).

It’s just… I want to write a novel! Or a short story collection. Something beautiful and creative. But my book outline is for something completely different.

I ought to write about gamification. And yet.

Through an iPhone, Squinting

If you want a vision of the future, imagine your arm holding up your iPhone — forever

It has been truly delightful to see all the imaginative augmented reality prototypes made by developers playing around with Apple’s new ARKit framework. It’s only been available for a couple of months, but developers have already gone to town with all sorts of fun ideas:

Amid the excitement, it’s easy to forget that we’ve been here before, many, many times. Back in 2015, Apple acquired Metaio, a German company that provided an SDK (software development kit) that allowed other developers to easily make augmented reality experiences. Two years on, ARKit is no doubt far more powerful and far easier to use than Metaio’s SDK, but the principle is the same.

And so are many of the applications. We’ve seen 3D objects superimposed on the real world on iOS device before, powered by Metaio:

And we’ve seen people plopping down inexpensive furniture into their homes before:

I don’t mean to rain on ARKit’s parade. The ease of use, lack of licensing fees, and sheer power means that we’ll be seeing a greater variety of ideas than we ever did in Metaio’s time, and so it’s entirely possible that someone will figure out an idea that makes phone-based augmented reality really take off. This ARKit-powered measuring tape prototype is actually very neat and useful:

But I don’t think this time is different.

All of these videos you see are incredibly misleading — not because they’re faked (they aren’t), but because they make it seem as if your field of view will be filled with the digital reality.

We usually don’t have to worry about this for videos taken by smartphones because in those cases, it’s actually true: when you watch a video on Snapchat or Instagram, you’re seeing what you’d really see if you were there. But when you watch an augmented reality video, you’re definitely not: instead, you have to imagine you’re holding up a phone at arm’s length, and seeing the video on that phone.

Looking at the world through a 5″ window is never going to be comfortable for longer than a minute. I’m sure there will be specific, short experiences like the measuring tape app that do well, along with some tourism and instructional apps, but I very much doubt we’ll see experiences even as long as 5 minutes, let alone 15 minutes.

As for games, so many of the prototypes are basically placing 3D objects on top of real world 2D planes, like your coffee table:

Forget about accessibility or comfort. I fail to see how this is more fun than a non-AR game that isn’t tied to a flat plane. It would be more innovative for AR games to involve manipulating of objects in the real world to influence the digital world, but that’s decidedly tricky when you’re holding up a phone or tablet.

Before you say “Pokémon Go”, let’s be clear — literally the first thing any decent player does in that game is turn off the augmented reality layer (where the monsters are superimposed on top of the real world camera view) because it eats up your battery and makes the game harder to play.

I don’t buy it. And I don’t think Apple does, either. Unlike Google’s shameful boosting of the dreadful Google Glass, Apple has thus far been comparatively quiet about ARKit. I’m sure they realise that most people don’t want to develop their shoulder muscles by using AR all the time.

No, this is all preparation for their future heads-up display — one that really will fill up your field of vision, be perfectly comfortable to use, utterly desirable, and only barely affordable.