We investigate hybrid user interfaces (HUIs), aiming to establish a cohesive understanding and to adopt consistent terminology for this nascent research area. HUIs combine heterogeneous devices in complementary roles, leveraging the distinct benefits of each. Our work focuses on cross-device interaction between 2D devices and mixed reality environments, which are particularly compelling, leveraging the familiarity of traditional 2D platforms while providing spatial awareness and immersion. Although prior work has prominently explored such HUIs in the context of mixed reality, we still lack a cohesive understanding of the unique design possibilities and challenges of such combinations, resulting in a fragmented research landscape. We conducted a systematic survey and present a taxonomy of HUIs that combine conventional display technology and mixed reality environments. Based on this, we discuss past and current challenges, the evolution of definitions, and prospective opportunities to tie together the past 30 years of research with our vision of future HUIs.

Survey Website

Augmented Reality (AR) interfaces support users by providing access to digital content within real-world environments. However, displaying content at the users’ eye level might result in the occlusion of the real world. Therefore, it requires finding AR content placement areas that free the users’ field of vision. In this work, we systematically investigate two content placement areas beyond the users’ eye level: the ceiling and floor. To understand how potential users perceive virtual content on the ceiling and floor and how the content should be placed on these areas, we conducted two user studies. While the first exploratory study showed the general usefulness of either area, the second quantitative study allowed us to define optimal placement parameters regarding visibility and comfort. With insights from our studies, we provide design recommendations for future AR applications that support 2D content presentation on the ceiling and the floor.

Video Figure | Video Presentation

Appendix | Supplemental Material

We present Marvis, a conceptual framework that combines mobile devices and head-mounted Augmented Reality (AR) for visual data analysis. We propose novel concepts and techniques addressing visualization-specific challenges. By showing additional 2D and 3D information around and above displays, we extend their limited screen space. AR views between displays as well as linking and brushing are also supported, making relationships between separated visualizations plausible. We introduce the design process and rationale for our techniques. To validate Marvis’ concepts and show their versatility and widespread applicability, we describe six implemented example use cases. Finally, we discuss insights from expert hands-on reviews. As a result, we contribute to a better understanding of how the combination of one or more mobile devices with AR can benefit visual data analysis. By exploring this new type of visualization environment, we hope to provide a foundation and inspiration for future mobile data visualizations.

30s Video Teaser | Video Figure | Video Presentation

Appendix