Introduction
Video games act as a social and emotional outlet for many people. However, most mainstream 3D video games remain largely inaccessible to players with visual impairments (PVI) because these games lack the necessary accessibility tools. Furthermore, games created specifically for PVI are greatly simplified when compared to mainstream games created for sighted players, and previous work has suggested that these games are not very fun when compared to their more mainstream counterparts.
We believe that an important challenge is to allow PVI to play the same mainstream games that sighted players play with the same amount of agency and fun afforded to sighted players. Doing so involves granting PVI with abilities that are equivalent to what sighted players have within games.
One key ability that a sighted player has is the ability to use the in-game camera to look around — to find items and otherwise interact with the world. This ability allows players to understand the game world around them to develop plans of action. Existing games, however, do not offer PVI with an analogue to the in-game camera that allows them to "look around." We believe that creating such a tool is critical to allowing PVI to explore 3D game worlds on their own terms and providing them with gaming experiences that are more equivalent to sighted players.
To this end, we introduce NavStick.
What is NavStick?
NavStick is a self-directed directional surveying tool that grants PVI with the ability to look around themselves within virtual worlds. NavStick works by repurposing the right thumbstick on a game controller to allow players to survey their immediate surroundings by "scrubbing" the thumbstick in a circular fashion. As the player "scrubs" in different directions (their 1 o'clock, 2 o'clock, etc.), NavStick uses spatialized (3D) audio to announce what lies in that direction via line-of-sight.
An illustration of a game controller. A user's hand is pushing on the right stick in the one o'clock direction; NavStick will announce whatever lies in that direction via line-of-sight.
At a lower level, NavStick represents the objects in the player's surrounding environment via a circular data structure called a "NavPie." The NavPie can be thought of as a circle or a pie where each slice represents an object that is visible from the player's current point-of-view. The direction of the slice represents where the object is in relation to the player at that moment in time. The size is determined by how big and how far the object is from the player. Tilting the stick then queries the slice (and, thus, the object) that lies in the direction that the stick is being tilted in.
Representing a player’s surroundings with NavPies. A player is surrounded by five objects (A-E) of varying sizes. The figure shows the player at two different positions among these objects. Top: Two diferent player positions among these objects. Bottom: Forward perspective NavPies, where each pie has been rotated such that it visualizes the pie from the player’s forward perspective. The arrows represent the direction that the thumbstick must be tilted in order to 'look at' each object.
NavStick's goal is to replicate the ability to look around by giving users on-demand, random access to information about their surroundings in all directions. That is, users can survey any direction instantly without having to traverse a list or other sequential interface. This stands in contrast to many tools in existing blind-accessible games, which have users survey their surroundings via linear menus or instructions, and which often generate unwanted and distracting sounds by outputting announcements automatically. We co-designed NavStick alongside a member of our research team who is blind.
User Studies
We performed two fully-remote user studies to evaluate NavStick. One to evaluate NavStick against status quo surveying tools for PVI, and another to explore NavStick's potential within a 3D adventure game. The following subsections give an overview of the studies and some notable findings.
Study 1: NavStick vs. status quo surveying tools.
Our goal with Study 1 was to evaluate NavStick with respect to the status quo means allowing PVI to survey an environment — traversing a list of items. List- and menu-based systems are very popular within games created for PVI; thus, we used this study to evaluate NavStick against menu-based surveying. To do this, we created a menu-based surveying tool called NavMenu. NavMenu presents an alphabetically-sorted list of points-of-interest within the area that the user is currently in. We compared both tools with each other and explored each tool's advantages and disadvantages.
A player using both NavStick (left) and menu-based surveying via NavMenu (right).
We had nine blind participants perform various tasks within a virtual grocery store aisle. These tasks involved participants using NavStick and NavMenu (in a counterbalanced order) to survey the eight items within the aisle in order to answer some questions. These tasks corresponded to Darken and Sibert’s classification of wayfinding tasks for virtual worlds. There were five distinct tasks. The currently active task dictated whether we asked questions about the content or the layout of the aisle, or whether we asked them the question before or after they surveyed it.
An overhead view of the Study 1 aisle. The character is in the center, and the blue arrows point to the eight item slots used in Study 1.
We also added a very simple video game level as our sixth task. This level was a recreation of a level from the classic blind-accessible game, Terraformers. Participants played through the level twice — once with each tool.
Comparison of Terraformers environments. Top: The room in the original game. Bottom: Our recreation of that room within Unity. The room consists of four objects: a computer terminal (#1) that reveals the location of a key (#2) which must be inserted into a keyhole (#3) in order to open an exit door (#4).
As for some of our notable findings: We found that participants felt that they were able to form a significantly better mental map and had significantly more fun with NavStick than with NavMenu. However, we also found that participants generally did not prefer NavStick for every task — participants preferred NavMenu for tasks that do not require a sense of knowing the physical direction that an item lies in, which NavStick is very good at communicating.
Study 2: NavStick within a 3D adventure video game.
Study 2 was an exploratory study in which we explored NavStick's potential in making 3D games made for sighted players more blind-accessible, and the extent to which NavStick works for complex situations found in modern 3D adventure games. To do this, we created a 3D video game called The Explorer, derived from Unity's 3D game kit.
A still shot from The Explorer, our prototype 3D adventure game for Study 2. The main character ('Ellen') is aiming a laser gun at an enemy (a 'Chomper').
Using The Explorer as our study's testbed, we had seven blind participants use NavStick to help them traverse a 3D game world that we created. The game world consisted of two large areas comprising a total of eight segments, or shorter levels. We designed the eight segments to possess environmental characteristics found in existing 3D games that could present a challenge for NavStick. These were enemy movement, time pressure, and environmental occlusions. Players simply had to reach the goal checkpoint of each segment, while avoiding obstructions and defeating enemies as they desired.
Overhead views of Segments 1-8 of The Explorer. Key: Yellow, labelled boxes indicate each segment. White boxes = pressure pads. Bright blue circles = checkpoints. Hollow orange circles = roaming enemies. Hollow red circles = stationary enemies.
As for some of our notable findings: All participants completed all segments of the game, and we witnessed how NavStick offered players with an enhanced sense of agency. Many participants commented on how using NavStick to "look around" allowed them the freedom to make their own decisions on how they wanted to traverse the level.
However, we also found that the various environmental characteristics that we integrated into the levels caused participants some trouble. The three characteristics — enemy movement, time pressure, and environmental occlusions — prompted progressively higher levels of frustration in participants, with levels featuring occlusion having some of the highest frustration scores, pointing to future work to improve NavStick or create new tools to solve these challenges.
Implications for Future Blind-Accessible Games
Our study results are promising for the prospect of making games mainstream 3D adventure games — that is, those created for sighted players — more blind-accessible. However, we see two main perceptual challenges that must be overcome.
The first deals with the fact that NavStick only announces objects within a player's line-of-sight. Yet, many participants desired some form of "x-ray vision" — that is, being able to see the world beyond their line of sight. Sighted players usually have access to a world map or a minimap that communicates information about the greater area they are in. So, how can one create an equivalent tool for players with visual impairments?
The other challenge deals with aesthetics. Many participants wanted to know what the overall world within The Explorer looked like beyond merely knowing what objects it contained. 3D game worlds are often richly detailed and fantastical, and The Explorer was no exception. So, how can we go beyond merely communicating points in space to communicating what the world actually looks like?
A scene from The Explorer.