Virtual Reality Investigations Microsoft, Xbox
Concept art depicting spectators within virtual reality watching an ongoing video game match.

Project brief and my role.

Over a year, I worked on a small team consisting of Joe Wheeler, a senior interaction design lead, Jeremy Frye, a motion designer, and Tyler Esselstrum, a technical artist. Our goal was to investigate best practices and opportunities in the realm of virtual reality. We explored concepts from sketching to building out prototypes in order to visualize and demonstrate concepts for discussion. My role was visual designer, illustrator, storyboarder, and at times, prototyper.

A prototype that asked the question, how would an Xbox interface work in virtual reality. How would scrolling work if there was a floor? Would we utilize peripherals in a user's field of view? Could z-depth be used as a visual cue for information hierarchy?

What is the best virtual reality experience for creating you--your avatar?

A virtual reality experience depends two things: immersion and presence. Immersion is the objective, quality of the sensory experience that reliant on the technology--such as the quality of the displays in a HMD. Whereas presence is our psychological perception of being or existing in a virtual space. The latter we can more easily design for. For example, an experience user that looks down and sees they have body has a greater sense of presence than a user who is just a floating ghost. So when proceeding, we refocused our question to "What is the most present virtual reality experience for creating your avatar?".

We first investigated other avatar creation experiences: Oculus, Google Daydream, Facebook VR, Rec Room, and an assortment of other social VR apps. And we were most drawn towards the experiences that featured a mirror that reflected you, the avatar, while you customized your appearance. This paradigm offered the greatest presence because it's a situation we're all familiar with: dressing ourselves in front of a mirror.

  1. Self presence: We can see ourself in the mirror which grounds us in this space.
  2. World presence: The mirror behaves how we expect in the real world.
  3. World and self presence: We can pick up clothing items and throw them around and they behave how we expect. We can also drag it to our body to dress ourself.

We explored other models by asking ourselves: What if what in the future we needed to dress our lower body too? What if there were such a large catalogue of options we needed more robust interface? What if someone with a physical disability needed to do this?

Mirror model: A mirror communicates that this avatar, what you see in front of you, is you. And you are changing your appearance by dressing yourself.
Doppelganger model: One step removed from a mirror, a scale presentation of you copies your movements. You can either dress yourself, or the doppelganger which you can rotate to see from behind and dress with greater ease, especially when it comes to lower body.
Doll model: A miniature scale version which copies your movements and is rotatable on every axis. Easiest to view from every angle, and easiest to dress. Furtherest removed from your identity.
A prototype of avatar creation. The challenge was fitting the menu and the avatar within the users FOV so that they could see the clothing items and the resulting avatar appearance with minimal head movement.

What is the best "home" for virtual reality?

When I approached the challenge of designing a hub area for a hypothetical "home" I went about framing the problem similar to how I would when tackling a visual design problem and defined our core question as "What is the most comfortable room we can make in virtual reality?" We set about first to define "comfortable" by gathering images of places and things that we would want to be this space or apart of this space. After creating moodboards for interiors and exteriors, we analyzed and sorted them into themes.

Interior moodboard: Light, simplistic architecture, lived-in elements, and nature.
Exterior moodboard: Emphasis of a large scale world, repetition that augmented that scale, nature, and proof of life.

We shifted our perspective to include an architectural approach and investigated prospect refuge theory, a psychological phenomenon which suggests that the spaces we find most comfortable present us with the greatest vantage point, with the least visibility. It is thought to be a biological response ingrained in us by ancestors who looked for places that provided good shelter from predators and weather, while providing a good vantage point to see prey and predators.

We find most comfortable present us with the greatest vantage point, with the least visibility.

From there I sketched a few proposals for a home incorporating elements from the moodboards, prospect-refuge theory, and competitive analysis. My initial ideas had overhanging roofs for shelter, soft-barriers such as water, and a single wall behind the user to provide the widest view, while providing protection from behind.

Concept sketches placing users at a high vantage point while providing shelter that obscures a minimal amount. Vines hang from above and flying cars pass by providing a proof of life and the presence of life and nature.
A later concept sketches put walls behind the user's back to prevent unease. Soft barriers, such as water or valleys, rather than hard barriers, such as walls or cliffs, were incorporated.

I paint painted a few concepts that iterated on different questions: Does it feel better your don't seem trapped on a platform? What if the time of day reflected the real world? Does there need to be more contrast between the UI and the world? Do we need real world objects in the world for reference of scale?

Water around the user works at a soft barrier, over hangs give a feeling of security without confinement, and buildings similar to the one the user is standing on extend into the horizon to inform the larger environment.
To create larger contrast between the interface and the environment, this concept put the user in a darker time of day.
Benches on the wall on the right provides a real world reference that helps to ground the user.
A simple villa surrounded by water with a hard barrier. A bridge extends to a far away island to convey scale, distance, and freedom.

What would be the "taskbar" of virtual reality?

We can categorize virtual UI into two buckets of interaction models: point and touch. Point models are more ergonomic, requiring only minor wrist movement, and they interface with UIs that are usually further away, allowing for greater content density. This model works for most interfaces. Whereas, touch, like wrist anchored UIs, are close, relative to the user, and work better for repetitive tasks leveraging muscle memory. The wrist mounted UI was a good model for a UI that persists throughout experiences, a perfect VR paradigm for the Xbox's Guide, a menu that could be pulled up anytime.

One challenge was designing for touch, without haptic feedback. We needed to rely on visual and auditory feedback to show proximity and activated states.

Various sketches for on arm interfaces.
Messaging: Voice to text concept.

When prototyping with the leap motion device, we ran into the problem of occlusion. The device could detect body parts accurately when two parts cross. This lead us to design for interfaces that avoid occlusion.

Volumetric UI: Cubes fill with color as pointer hand moves closer. Activation occurs on contact.
Planar UI: Planes are pushed toward outlines. Activation when planes contact outlines.
How would the guide work with other devices?

See more projects.

Virtual Reality InvestigationsMicrosoft, Xbox UX Design/Visual Design, 2016-2017
Hometown LiveSinclair Digital UX/Visual Design, 2014
Project Scorpio Microsoft, Xbox Visual Design, 2016-2017
Axon Investor RelationsTaser/Axon UX/Visual Design, 2015
2 HopsPersonal Game Design, 2014-2015
Various PaintingsPersonal Digital Art, 2009-Present