By Yijian Lan & Alexander Kübler — Interaction Design Project. Time Frame: Mar.17 – Jul.17. Visit Medium to see more.
Discovering Designprinciples For Augmented Reality Applications
The technology Augmented Reality and the associated design and development processes are still in their infancy. Currently there are few to no design principles or guidelines for goal-oriented work in an augmented world.
There are a variety of applications and devices on the market. Some of them offer a new experience and are innovative at first glance. In the long run however, none of them offers any real social value which optimally utilizes the technological state. The use of technology is currently benefiting mainly the B2B sector. We as customers and consumers are completely ignored. Therefore it was important for us to find an area from which the society can benefit as well.
Marnix Kickert–"Conquering Spatial UX is one of the most exciting challenges designers will face the coming years."
Apple’s AR bet still has a lot to prove (Image: Apple)
The core of our project „AR-Port“ is the technology „Augmented Reality“ and its design processes. For us Augmented Reality (AR) is the superposition of two- or three-dimensional information and elements that interact with the physical environment. We looked at the question of what AR is and what opportunities this technology offers and will bring in the near future. In order to research design principles for AR applications, it was important for us to find a clear use case to search and develop AR design principles.
ARport is a holographic navigation application that acts as an intelligent companion, guide and coordinator on air travels. The holographic content is displayed via a AR-Head Mounted Display. Depending on the situation, the extended information is contextually projected into the user’s real environment.
For the Microsoft Hololens, we have developed a working prototype in Unity 3D that offers live and 3D map navigation, a language assistant and a dashboard with all the important user information.
- Dashboard– The dashboard can be used to access all important contextual user data, such as schedules, navigation routes, tips, and environmental information.
- Map and Navigation–The 3d environment map makes it easy to create routes, search for points of interest, and retrieve relevant information.
- Voice Assistant–The Voice Assistant is easily controlled by the language of the user and serves as a supplementary input and search method of the application.
Our design process is divided into four main points: 1. Research, 2. Synthesis, 3.Ideation and 4.Development.
The development of holographic AR applications and their design process was new and time-consuming for us as interaction designer. In order to make the development more efficient, we have put our focus on fast and iterative testing. Smaller ideas were tested directly according to sketches, which were later developed further and optimized. We were able to build the information architecture bit by bit and implement and test the user requirements of our application.
We have found our own software workflow that allows us to test our design efficiently and iteratively. With box and OneDrive we can open our screen design directly on the Hololens. Hologen is an app for sharing holographic content. We are working with it to quickly provide our 3D models for the Hololens.
Software workflow for the Microsoft Hololens
- Yijian Lan (Strategic & Interaction Designer);
- Me (Strategic & Interaction Designer)
- UI/UX Design
- (User) Testing
- Documentation & Layout
- Experimental approach
- Prototypical work process
- A lot of user testing
- Collecting design principles
- Experienceable application
Our Project Process
We started with the investigation of potential fields of application in which augmented reality provides real added value.
From the beginning of our research we were able to summarize 16 potential fields of application with the help of the „Goldman Sachs Group“, Microsoft, the Deutsche Bank AR study and other sources. These include: Architecture, Exhibitions, Education, Coworking, Healthcare, Engineering, Journalism, Live Events, Marketing, Medicine, Navigation, Retail, Real Estate, Tourism, Videogames, and Video Entertainment.
Datasource: Goldman Sachs
For 10 selected application fields we have defined different subcategories. In doing so, deeper coherent topics and areas were considered. At the same time, we discussed the possibilities of the new fields. When considering the fields, we were interested in the degree of innovation, the standalone features (why AR?), the potential for solving problems, and the ability to design content.
Finally we decided in favor of indoor navigation. Our project scenario focuses on the context of airports and their infrastructures. As navigation technology is relatively well developed and evolving rapidly AR navigation can be achieved very quickly. For the development and the seamless use the correction of the current technical performance problems is necessary. Because we already use the navigation apps on our smartphones every day the navigation via augmented reality has the potential to become a normal scenario. We believe that navigation can be one of the first areas where augmented reality is really brought to the users.
According to the research and our project insights, two favorite areas were selected for further processing on the basis of the collected „opportunity areas“. With the help of How might we questions central questions of the areas were worked out.
From the „Opportunity Areas“ the points „And now?“ And „Transport routes“ were chosen. These two points together provide an optimal balance between the project topic of „navigation“, responding to different user needs and the inclusion of social and creative factors. The asked HMW questions should be seen as kind of objectives or guiding questions within the project.
4. Ideation & iterative Design/Prototyping
In the next step, several iterative brainstorming, sketches, design and testing sessions in the team were run through based on the „How might we?“ questions. As many ideas as possible were developed in short phases.
In order to generate the widest possible range of ideas, the iterative phases of ideas were repeated several times at each point. In the course of the ideation phase, around 10-20 ideas were generated for each question of the two „opportunity areas“. The iterative ideation processes serve above all to expand the horizon of ideas and thus to find important concept components.
Design and 3D Modeling
5. Final Design
Our finished application is displayed without background on the Microsoft Hololens. The blue background of the images and content shown serves exclusively to illustrate design content and details. The finished design of our dashboard consists of three essential information levels.
- The main menu items (Map, Voice Assistants, Settings)
- The contextual information, which is located in the middle of the screen and can show different information depending on the travel process or travel section.
- Additional information such as, current position, time and date, and status bar and the display of active navigation routes together.
Key takeaways–Here are a few of our design principles, insights and tips we have worked practically and iteratively within our work. They serve as clues and aids, which should support designers of all kinds in setting up and developing an AR application in a goal-oriented manner. We hope that these insights are helpful.
- From Smart phone to HMD — The design for head mounted displays and for the smartphone is fundamentally different. When designing a user interface for a smartphone, the screen of a smartphone is the canvas. To some extent, the designer has a great deal of freedom with this canvas. Nevertheless, a screen is limited to its image area. In contrast, augmented reality ideally uses the entire environment. The most important thing is that digital content and reality overlap. This is an advantage, but also a disadvantage. Designers can put any content into the real world and everything could be interactive. It should always be considered how digital content is presented to users? When is the right moment for holographic content and when can digital content disturb or even endanger the user? In other words, as a designer we should handle the placement of AR Content conscientiously.
- Read the Guidline ! — This is not really a new story, but it is so important. Guidelines help us save a lot of time, allowing us to make quicker decisions. From an official guideline for our target platform, we quickly learned many basic design principles. Some basic design principles that we already know can be applied to any AR device, others are heavily dependent on the hardware parameters.
- Know your new device and do many tests —In our project the Microsoft Hololens is our target platform. There are so many factors that differentiate the device from conventional devices. For example, the hololens uses an additive display. This means that the light used to render the holograms is covered by the light from the real world. So when we design a graphical user interface on our computer, we have to keep that in mind. The designs we design on the computer always look a bit different on the Hololens. So before you start designing, we as designers should read the official documentation and design guidelines. But you can never be sure. The only solution is to test, test, test. Furthermore, it is important that black can not be displayed on an additive display, whereas white appears very bright and can really dazzle the user. Last but not least, the hardware limitations can not be ignored. For example, the field of view of the Hololens is only 30 degrees. If you do not adapt the design to these limitations, important information can be lost. This can potentially damage the user experience and even be dangerous in daily use, depending on the context.
Thanks for your interest–Further help and tips will follow shortly. If you have questions and are more interested in our project feel free to send an email or contact me on LinkedIn.