And Yet It Moves is a physics-based platform game in which players rotate the game world at will to solve challenging puzzles. We released it for PC in April 2009 with common keyboard controls.Players could rotate the world (only) in 90 degree steps by pressing buttons. Bringing And Yet It Moves to the Wii meant one thing: adapting a game designed for digital input to the wide range of analog values of the Wii Remote. This turned out to be quite a challenge in which we invested a lot of time and energy. Here’s what we learned while doing so:

Ditch the thresholds and adapt your gameplay to the controls!

All of And Yet It Moves was designed for digital input: the 90 degree limit, the levels, the challenges, etc. This is why we were always cautious when people were enthusiastically telling us how cool they think it would be to play our game with motion controls. In the end they were right. And Yet It Moves is now a game that plays better with the Wii Remote then with buttons. But it took us some time until we got there:

What didn’t work

The first thing we, programmers, think when reading in the wide range of analog values the Wii Remote sends to us, is: “Well that will be easy, we just wait until x is higher then 2.5 and then we trigger this function”. This is our digital-input-state-of-mind. We try to map movement to the on/off style of digital input, because that’s how we are used to think about controls and that’s how the code already works most of the time.

This is also how we, Broken Rules, started with And Yet It Moves: we looked at the rotation of the Wii Remote, calculated an angle and if that was higher/lower then a certain threshold we triggered the same function the buttons did. We had this fairly quickly and we even got used to playing with it, although it was always imprecise and prone to errors. In our playtests people got frustrated fast and always asked for button control options. Not good!

Thresholds

Our approach of movement detection was based on thresholds. Nowadays I always get cautious if a motion control implementation is based solely on thresholds.

While the Wii Remote is very responsive, it’s also very ambiguous. Many movements can trigger the same set of data, because all the accelerometers measure is an acceleration, or in other words, a change of velocity. Lots of movements can output this change lead to the same results, so you can never be sure what really happens.

This means that a simple threshold implementation will always be prone to errors. Especially quick & hard movements or jerks, can be a problem, because due to the inertia of the accelerometers, you will always get a very high acceleration followed by a high deceleration. You algorithm has to take care of these high peaks, because otherwise it will just interpret them as a big rotation. Just ignoring them is an option, but it also introduces a lag to your controls.

The drifting Origin

Thresholds also always depend on how people hold the Wii Remote during gameplay. We noticed that in general people are not aware of how the Wii Remote is orientated in their hands until they look at it. For example after doing a rotation they think that they rotate it back to a horizontal stance, but they actually hold it at an angle. The more they play and the more challenging the situation is, the more it will vary from the original horizontal stance.

We worked around this problem by making them press and hold a button and then start tilting the Wii Remote. This way we could use the moment of the button press as the original stance. While this worked most of the time, it still failed too often. Especially when doing multiple rotations shortly after another, this origin drifted so far that the bone structure of the players hands didn’t allow for another rotation anymore and the players where stuck.

Exceptions

To deal with the ambiguous data and the problem of the drifting origin, we added more and more exceptions and special thresholds to our algorithm. But the more special cases you include, the more inaccurate your detection becomes. And the harder it gets to debug and interpret wrong player movements. While the core movement a player has to do for And Yet It Moves seems simple, the special cases still became too many. We tried to filter double rotations, quick&hard jerks (maybe better: movements? doesnt sound like idiot that much :)), relative starting positions, etc. and every time we added a new case, an old one didn’t work as well anymore. Your algorithm also gets slower and less responsive the more data you filter out and the more you try to avoid false detection.

The missing feedback

The problem of the drifting origin, which I mentioned earlier, actually belongs to the fact that small movements with the Wii Remote give very little tactile feedback. When you press a button you feel it’s friction, you feel how long it takes to be down and thus quickly learn to have precise control over your actions. While the bone structure of the hands does provide some feedback for motion control, this feedback is only strong enough at the limits of its joints. You can use it at the extremes, but for the small movements in between, it gets difficult. Actually the only way to get feedback for these small movement is through your visual senses.

Because we don’t want our players to look at their hands while playing, we tried adding a visual feedback through a HUD. This way we could show the players on screen how they hold the Wii Remote, where the thresholds for rotating the world are, what happens when they do quick & hard movements, etc. This definitely helped a bit, but HUDs have one problem: players don’t look at them. Especially in more difficult situation where they get nervous and have to act fast, they tend to ignore all extra information. All they look at is the core of what is happening. In our case the player and the surrounding parts of the world. We did place the HUD right on top of the player, but they still ignored it, once the game got more intense.

What worked

Although we reached a point where we – or any player who invested enough time into learning the controls – could play And Yet It Moves using the motion sensors of the Wii Remote, we where never really satisfied with them. It was still better to play with normal buttons, and this meant it’s probably a better idea to ship the game without motion control support.

Adapting the gameplay to the controls

The turning point for motion controls that actually make sense in And Yet It Moves, has nothing to do with the detection algorithm. It happened when we took a step back and looked and the problems we noticed in our play-test-sessions. The main problem was the missing link between the rotation of the Wii Remote and the rotation of the world. The first is continuous, smooth and a bit different for everyone – it is analog – and  the other was stepwise, normed and perfectly tuned for digital input. Even if we had a perfect detection algorithm and the best HUD, the rotation of the world would always be an abstraction of the real rotation. What we needed was direct mapping and an immediate, in-your-face feedback of the rotation of the Wii Remote. When you think of it this way, it is perfectly obvious that we needed to adapt the gameplay, that is, we needed to get rid of the 90 degree step-wise rotation.

Once we allowed free rotation of the world, And Yet It Moves made sense on Wii!

Give direct feedback

With this simple – but tremendous – change in the gameplay we also got rid of our dirty threshold code with all its exceptions. We can use one simple & clean formula to calculate the rotation of the Wii Remote and use this to rotate the world. We don’t need to specially deal with quick & hard movements, etc., all we need to do is show it to the player. This means that high accelerations and decelerations – which happen when players do extra hard movements and/or quick stops – are directly mapped to the world rotation, resulting in short chaotic moments. But as soon as the Wii Remote is stable again, the rotation will be correct and logical and the world will be exactly rotated like the player wants it to.

Of course we could have tried to filter these out again, but that would have only resulted in lag in the controls and unwanted behavior at other situations. It proved that through the immediate feedback players see what happens and understand the correlation between their hard movements and the chaotic rotation of the world in-game. We can leave the exception handling to their brains, and human brains are way better at dealing with analog values then computers!

Pause everything else

But we could only leave these short chaotic moments in-game – and at the mercy of the brain of the player – because to rotate the world, players have to press and hold a button to freeze everything. The in-game character wont move, it – and all other objects – will stay at exactly the same position and players can completely focus on the rotation of the world – and build hypothesis on what will happen when they unfreeze the world after rotating it.

This pause enables players to understand the mapping of the Wii Remote to the world, deal with small hiccups of the mapping and gives them the peace of mind to do slow, smooth and exact motions. It removes the stress and converts hectic moments to controllable and solvable situations.

This freeze was in And Yet It Moves form the beginning, and is central to our physics simulation and gameplay, but it proved to be extra helpful for the motion control support.

Smooth the data

We did filter the data in one particular way, though; we smoothed it out. As I said before the Wii Remote is very responsive, and small but hard movements can output quite big jerks. It proved to be a great idea to buffer the data and always take the average of the last 20 acceleration values. At the beginning we were afraid that this might lead to a lag in the controls, but the output frequency of the Wii Remote is so high that it was no problem at all.

Take aways

From our experience of developing for the Wii Remote I say: avoid algorithms that depend too heavily on thresholds. (It can work, but only if the game needs to detect one kind of movement at any given time). At least be cautious, because heavy threshold use, is always a hint that you might still be designing for digital input. Mapping analog values to digital controls is like trying to put square blocks into round holes. And it’s actually a waste of data!

As a designer we need to be aware of this and adapt the game to the controls. Look for mechanics that naturally provide clear, instant and in-your-face feedback to the motions your players are acting out. Mechanics that can take a wide range of analog values as input, and that clearly map the real-life motions to the in-game action.