Design for Emerging Media

By Marcus on 11. Oktober 2017 — 5 mins read

Reading about Mixed Reality frameworks and models while dreaming of Madeira and ICIDS 2017.

There are a lot of academic conferences out there. At a lot of beautiful places. The combination of nourishing brain food throughout the day and a proper view in a decent hotel room overlooking the sea in the evening and the early morning sounds like the perfect combination to me right now. And – even better – you do not have to sell stuff apart from your idea and you academic persona, but no need for dull business talk.


Right now ISMAR 2017 is happening in Nantes: The IEEE ISMAR is the leading international academic conference in the fields of Augmented Reality and Mixed Reality. ICIDS 2017 is taking place in Funchal, Madeira, Portugal November 14-17: ICIDS became the premier annual venue that gathers researchers, developers, practitioners and theorists to present and share the latest innovations, insights and techniques in the expanding field of interactive storytelling. And RE:TRACE the 7th International Conference on the Histories of Media Art, Science and Technology will happen in Austria, November 23rd to November 25th.

I will be attending RE:TRACE after having read a lot of stuff about media archaeology and after having joined the Media Art Histories RESEARCH FB group while still reading Oliver Graus: Virtuelle Kunst in Geschichte und Gegenwart. I will definitely miss ICIDS 2017 and that is a shame, especially after having conducted a workshop there in 2016 without recognizing that Rebecca Rouse was there too.

Barba and Rouse

Together with Evan Barba she will present a paper called „Design for Emerging Media: How MR Designers Think about Storytelling, Process, and Defining the Field“. And i would so very much love to hear about that. Right now Rouse and Barba are amongst the very few that exactly tackle my field of interest. They both represent a relevant part of academia that is future-oriented and capable of pushing things forward in the „wet clay“ (Helen Papagiannis) phase of MR. And while Papagiannis does a good job of giving a short overview, Rouse and Barba dive way deeper in order to e.g. „derive a new definition of MR centered on human experience“ in the times of the „second wave of AR“ (Papagiannis).

For example Rouse will publish an article on „VR and Media of Attraction: Design Lessons from History.“ In Virtual Reality Programming Gems, Ed. William R. Sherman. Taylor and Francis. Forthcoming Winter 2017 while having published another paper on „Media of Attraction: A Media Archeology Approach to Panoramas, Kinematography, Mixed Reality and Beyond.“ Eds. F. Nack and A. S. Gordon. Interactive Storytelling: Lecture Notes in Computer Science 10045, Springer Press International. pp. 97-107 in 2016.


It feels natural and obvious that Rouse and Barba work together after Rouse amongst others built up on Barbas text „A Scale Model of Mixed Reality“ (2011) in 2015 with „MRx: an interdisciplinary framework for mixed reality experience design and criticism“

Here are a bunch of quotes after having somewhat cross-read both articles (2011 and 2015) mentioned above:

* Relevance: MR experiences „are receiving new attention from researchers in communities outside the core technical community that has traditionally done MR related research“ (Barba 2011, 117)

* Relevance: AR and MR „have captured the interest of a variety of fields in the humanities, arts and design. Each field brings its own history, vocabulary and perspective to bear on design and criticism. (Rouse et. al. 2015, 176)

* Motivation: „Such growth is a healthy and necessary step in the evolution of new media, and has historical precedent in the adoption of other media forms associated with film, television, and the web (Barba 2011, 117)

* Motivation: We argue for the need to rethink the technical focus of Milgrams definition (Rouse et. al. 2015, 175)

* Approach: The current model of MR experiences is rooted in its history as a collection of technologies. Hower, MR begins to play a larger role in mediating human experience. (Barba 2011, 117)

* Status Quo 2015: Barba et. al. have presented a famework – „a starting point for a humanistic engagement that we will extend (Rouse et. al. 2015, 177)

* Approach: Questions like this should be addressed „How do the media forms of AR/MR relate to former media practices, such as writing and performance? How do these new formats extend, reenvision, or supersede former practices?“ (Rouse et. al. 2015, 176) #mediaarcheology or „What are the social and political implications of AR/MR as an extension of other formal or informal social groups, etc.“ #society

* Approach: We identify the concept of spatial scale as a central, and largely unconcious, component of MR (Barba 2011, 117)

* Background: Milgram and Kishino: MR == continuum between two extremes of purely physical reality and completely virtual reality, importantly characterized by their method of „display“, usually visual. Foundational work but a number of shortcomings… The conception of MR must be broadened… (Barba 2011, 118)

* Background: Majority of previous work can be divided into two categories 1) research on enabling technologies, computer graphics and tracking and 2) evaluation of users abilities to accomplish tasks. Such work does not fully engage the experiential dimension of the technology and how it fits in the wider context of everyday practices. (Rouse et. al. 2015, 176)

* Definition: From Azumas definition, that specifies three criteria…to a working definition of MR with following five elements: mapped, embodied, spatial, temporal, framed: Mixed Reality maps physical and virtual elements into a hybrid frame of reference that mediates interactions in time and space (Barba 2011, 120)

* Definition: „Milgram´s and Azuma´s technical definitions were developed in a period when AR was an experimental technology. They are not suited to capturing the experiential aspect of AR or MR as they move beyond the research lab and becomes part of everyday media culture.“ (Rouse et. al. 2015, 177)

* Milgram and Azuma are centered on the screen, understood as a technology for combining two radically different types of information (virtual and physical). The Barb-MacIntyre approach is centered instead on the user as a perceiving and embodied subject. They have given us a cognitive lens through which to measure MR experience design. We will propose three additional lenses. We extend Barb-MacInyre to take into account the cultural and social dimensions of MR experiences. We focus on the quality of the experience and only secondarily by the mediating technology (Rouse et. al. 2015, 177) Short intermezzo: What would McLuhan say?

*MRx instead of MR. MRx are hybrid, deeply locative and often site specific and esthetic, performative and/or social. Our interdisciplinary framework is not intended as a replacement but rather as a complement to the technical work or AR and MR computing communities. In order to get a more nuanced understanding. (Rouse et. al. 2015, 180) Short intermezzo: Okay, but is it useful?