On the commute to work – you’re on your phone, in an uncomfortable subway seat. At the office – at least two monitors, and a chair that’s set up ergonomically… at least until your co-worker borrows it. Back home – in front of the TV, on the sofa after a long day at work.

We go from screen to screen, from chair to chair. It’s problematic, and we know it. Sedentary behaviour is associated with negative health outcomes that are different from the ones just resulting from lack of exercise – i.e., you cannot fix it with the occasional visit to the gym. So what do you do when the boss doesn’t want you to take a walk around the block on company time?

It’s not easy to change the sedentary workstyle widely thought to be inherent to desk jobs. Ideally, you’d get rid of the monitors binding you to your desk, but that would mean losing sight of your e-mail inbox, IDE, or design tools too.

Or would it? What if we tell you that you can do your job without your desktop computer, and transform your work tasks into ones that allow you to get up and move around?

With augmented reality glasses, we’ve made this a possibility. Imagine coming into work in the morning, and instead of starting up your computer, you put on your AR headset. You’ll be able to see all the data you need right in front of your eyes, no matter where you are. And not only does this make it possible to move around while you work, we’ve integrated the essential physical activity into your daily routine, replacing traditional work tasks with active ones. This will give you all the health benefits you need without taking a toll on your productivity.

We’ve drawn from the playful world of sports to shape the office of the future, for which we’ve augmented three common office activities: logging in, sending an e-mail, and lastly – for times when you’re still sitting down for too long – taking a short break. With a physiotherapist’s advice, we have ensured that the movements the new task designs elicit will be especially effective in preventing the physical complaints that office workers often fall victim to.

Check out the video below to see what it looks like, and keep reading to find out more about the technology, our prototype, and the design process.

For The Techies: Meta 2

To realise all of this, we’ve used not just any AR glasses, but interactive ones; they allow you to use your hands to interact with all the AR elements you see around you, as if they were really there. This technology is still in its infancy, but already widely popular thanks to Microsoft’s Hololens and Metavision’s Meta 2. The Meta 2’s unique 90-degree field of view lends itself especially well to creating a realistic, immersive experience, and along with its helpful online community and the fact that its interactions with AR objects are highly similar to those with real-world ones (encouraging intuitive understanding), we saw the most potential in this device for developing our active AR workspace.

AR is going to happen in a big way. “We will wonder, when it does, how we lived without it. Kind of how we wonder how we lived without our phones today.”
— Tim Cook, Apple CEO

The Meta 2 was released to developers in late 2017. It offers an SDK for Unity with various ready-to-use interactions, such as a grab interaction that allows the user to move an AR object around by grabbing it with their hand. There are also interactions for rotating and scaling objects. You can extend or modify these scripts to create your own. We prefer these kinds of interactions over the Hololens’ interactions that allow you to manipulate objects from a distance; to make a product intuitive and usable, it makes sense for AR objects to follow the same rules as real objects, meaning you can only interact with them when they’re within reach. For the same reason, AR objects should resemble their real-world counterparts, and 2D GUIs should be avoided in the 3D environment.

The Meta 2 uses several cameras and sensors embedded in the headset to scan the environment and track users’ hands. Environment mapping does not register flat surfaces very well yet, making it difficult to position AR objects ‘on top of’ real-world ones like a desk, something the Hololens does allow. Hand tracking also leaves something to be desired, as users need some guidance to execute actions in such a way that the device ‘understands’ them – and even if you’re doing it by the book, your actions aren’t always registered. However, during the six months we’ve been working with the Meta 2, we’ve seen several SDK and firmware updates and have already noticed significant improvements in both environment mapping and hand tracking.

The Active AR Workspace

1/3: Logging in. First things first: to get started in your new work environment, log in to the system by entering your password. Instead of using your keyboard, you’re going to score a hole-in-one by solving this AR combination lock. It’s at about eye level – so don’t sit down!

2/3: Sending an e-mail. Once you’re logged in, the workday starts. As what is perhaps the most often-used communication channel at the office, e-mail is the next task we’ve augmented, and we’re going back to the past with the technology of the future: grab a piece of paper, type up your message, get up and throw it through the AR basketball hoop to send it to your co-worker.

Throughout the different tasks, an animated basketball character is there to guide you through it all, get you familiar with the new technology, and help you out when you need it. Once you get the hang of it, he’ll leave you to it.

This may look like an exciting concept, but are you going to have to do this every single time you want to message someone? Well… that’s up to you. You can set your own goals and decide the frequency with which you want to get some movement in.

3/3: Taking a break. If you’re not meeting your goal and have been sitting still for too long, we’ll let you know – not like that annoying pop-up tool that comes preinstalled on so many company-issued laptops, but in a gentle, non-obtrusive way. And instead of forcing you to take a break, we’ll give you something fun to do for a minute or two: in a short AR game, you’ll be able to compete with your co-workers while getting the activity you need.

For the version of our product we’re showing you here, we’re staying in line with the core value and goal by giving it a sports theme – but once the hardware has become commonplace enough that an app like this can be brought to market, there will be a plethora of skins to choose from. Maybe you’ll make your office feel like outer space with a Star Wars theme, or find your inner peace with a calm and minimalist style.

This customization is just one of the ways in which we want to keep users engaged. Throughout our prototype, we’ve incorporated familiarity and fun. Building on the Meta 2’s natural interaction style, we’ve made sure that the AR objects (like the clipboard and basketball post) look like the real thing, so users intuitively know how to use them. The gamification aspect also serves as a way to motivate users to keep using the technology to its full potential, and can be expanded in the future, for example by setting up competitions between departments.

A Digital Carrier Pigeon: The Design Process

The process of designing and developing an AR work task is a fun challenge. Because a digital 3D environment offers an entirely new way of interaction, you cannot simply apply 2D design guidelines to it, and not a lot of research has been done on the topic yet. To help developers get started, Meta offers some spatial design guidelines which have been extremely useful to us, and we’ve looked to experience design and storytelling to make our prototype more engaging and coherent. This is also how we arrived at the sports theme – initially, the three tasks were just that, each one with its own style. This is what the messaging concept initially looked like:

From our consult with the physiotherapist, we knew that large gestures executed slowly as well as standing up are effective movements to prevent common complaints that result from long periods of inactivity. This is why we require users to grab a paper and put it in front of them; to fold the paper in half; to add a stamp; and to stand up and lift their arm. The bird seemed perfect here: it resembles a real-life thing with a similar function (a carrier pigeon) and it makes sense for it to be above user’s heads, giving them a real reason to get out of their chair. (And of course, an API was used to really send out the e-mail you typed up!)

For the sports redesign, we had to kill our darling and get rid of the bird – which meant we needed to give users another reason to get up, and a sports-themed way to deliver a message. We thought about an AR baseball bat to hit your message in the direction of the recipient, but due to the limited hand tracking of the Meta 2, this was infeasible. It would also be counterintuitive to crumple your message up into a ball when it still needs to be read by someone – this is why we didn’t want to make users throw their message through a basketball hoop, either. Instead, we decided users could give their folded message to the animated basketball character that’s already part of the prototype, and throw him through the hoop. He would then deliver the message to the recipient. The paper tray was replaced by a clipboard (like one sports coaches use) and the stamp was removed from the scenario; users now make a similar movement when giving the message to the basketball.

A similar process was followed for the other two parts of the prototype: recommended movements were central to the task design, it was skinned to fit the sports theme, and we tried to work around the technology’s imperfections as much as possible.

It’s Got Potential

The feedback that we received about our prototype during user testing was highly positive. Many people who tried out the AR workspace told us they would like to use it on a daily basis, and not just because it’s new and fun; they consider it a viable solution to the problem of their sedentary workstyle in the long term.

So definitely at work, but everywhere else too, interactive wearable AR offers a hugely promising way of mixing the digital and physical environment, by using natural interaction to control digital elements. Along the way, design guidelines need to be determined and validated, technology needs to be improved, and social implications should also be considered (if everyone’s reality is augmented, this may change the way we interact not only with technology, but with each other).

Eventually, we won’t even need screens anymore – not just the desktop monitor, but phones, tablets and laptops will become a thing of the past. Everything will be happening right in front of your eyes, as augmented reality changes the way we work, live, and play. We’ve started it at the office – but prepare for it to take over the world.