Mirage – Interactive Hologram Experience
Team Project | Interaction Design

Mirage – Interactive Hologram Experience
Team Project | Interaction Design

Quick Facts:

Role:

Interaction designer

Time:

Fall, 2014 (7 weeks)

Client:

Coachella, music festivals’ concert promoters
(05-711: Designing for Interactions | MDes Studio I)

[Skills]
Methods:

Guerilla research, Focus setting, Opportunity mapping, Directed storytelling, Affinity diagramming, Ubiquitous computing, Personas, Scenarios, Speed dating, Video sketch

Tools:

Photoshop, Premier, AfterEffect, Pen & Paper, Illustrator, Audacity

Deliverable:

Video sketch introducing the intelligent environment, Speed dating and detailed insights, Product pitch

Quick Facts:

Please see previous slide

Design brief: Develop and evaluate concepts for intelligent environments – spaces that intelligently sense what is happening and provide relevant feedforward and feedback to people. Leveraging breakthrough in machine learning, UbiComp sensing and sense-making, and crowd based intelligence, the design should make the invisible visible, and they should make machine intelligence understandable to people.

Smart Conference

Design an intelligent system that transforms Large-scale festivals into a living social entity. Envision what can be sensed, who would benefit from this information, and how it can be communicated across large spaces. Think about compressing time and place as well as communicating awareness of place.

The challenge here is to work in emerging design spaces where there’re no patterns to guide interactions. We need to be really creative, but still pragmatic.

Our solution: Design an interactive hologram concept that extends the exclusive concert experience from the artist to larger scale audience at Coachella (or similar kind music festivals), and enables the crowd to coordinate in sync with the artist and other attendees together. Concert goers will also have access to videos and photos from their personalized view, so that they can have a truly shared experience.

Crowd coordinating in sync with the artist & personalized footage to share

The Mirage system consists of two main parts: dynamic gesture tracking & projection mapping, and cameras combined with location-based recognition through bluetooth. Concert goers’ gestures will be sensed by depth camera over the crowd using infrared technologies. This information is relayed to a specific set of holograms that will be co-created by the artist, and displayed through projection mapping on fogs to create the illusion of 3D volumetric views for the audience. Cameras around the premier of the tent will take videos and photos during the performance. There are also bluetooth beacons that sync up with concert goers’ phones to recognize their location. Using the locations, videos & photos will be taken with respect to personalized views and later to be delivered to concert goers through the Coachella App.



 

Preliminary research & 1st round interviews

Focus setting and directed storytelling

We want to understand the root desire for why people attend and how to use that to create more meaningful experiences, so we sat with previous Coachella attendees for directed storytelling interview sessions. They are recruited through Twitter, Facebook by tracking hashtags with “coachella”, and through personal connections.

We also found it helpful to ask interviewees show photos or clips they took and talk a little bit about them, which are the memories that they capture so as to relive later. They are all very excited in being involved talking about Coachella experience as one of their most memorable experiences with us.

In order to familiarize ourselves with the context of Coachella, we also read through a breadth of literatures to understand the culture of Coachella, which evolves from a previously well-known trend of “.Com” culture that denotes an effect assembling long-tail consumers of different types of music & arts. One of our interviewees who actually attended the first Coachella at 1999 had also pointed out the essence of the music festival spirit has made him wanted to explore more at Coachella.

Design ideation

Summarize posible design directions

Based on our 1st round of research, we organized our findings and grouped them into valuable themes, using affinity diagramming method. Through groupings and re-groupings, our understanding of the design space gets wider and more consolidate, not hanging up on any one of the directions. This helps to get our hands wet around user desirabilities in the context of Coachella, so as to map design opportunities with them.

Different possible ideas based on user research

(Please see the 40+ ideas list to the right)

Groups of desirabilities are formed in the following three spaces:

  1. We wanted to explore ways to make the concerts more of an exclusive experience. Because of Coachella’s large influence on social media, there is a metaphorical bubble around Coachella of being there and experience it V.S. perceiving it from the outside on social media. From our user research, we found that there is a lot of FOMO regarding Coachella.
    FOMO means "Fear of Missing Out", which is an anxiety a lot millennials experience regarding missing out on something great.
  2. We wanted to explore ways to make more shared simultaneous experiences. From the user research, we found this event served as a mini reunion for people who lived apart. So we wanted to create opportunities that built on shared simultaneous experiences and created shared memories.
  3. We wanted to created opportunities for concert goers to participate and co-create their experience. From our research, this was an new interesting direction Coachella has been taking for years borrowing concepts from Burning Man (another music festival) with more interactive art installations.

 

Speed dating

Create scenarios to rapidly test and see how to connect to the REAL product opportunities

As we took the risk to make a very new service, where there’ re no social mores to inform us as to what people might desire or might find socially unacceptable, we want to gain some insight on the real opportunities, and make sure we are not misleading ourselves to the wrong direction. Therefore, we picked 7 of our design ideas from last round of research, and converted them into easy-to-test scenarios to perform speed dating with users. These scenarios are narrated by us, but shown on their own storyboards to hand to users, in order to help users connect current experience to the future.

During the speed dating sessions, we performed needs validation and user enactments. We watch and listen their reactions while talking about each scenarios, and then dig on triggers of certain desires that our designs were trying to resolve for.

It takes time, especially for user enactments, but making the wrong product takes more time.

 

Download our storyboards & lead questions

Three possible directions

Pull up the top 3 provocative ideas

Asking previous Coachella concert goers during the speed dating sessions about “will you pay extra money for a service like this”, we found that there are disconnected parts between the observed needs from our 1st round user research and the perceived needs from user assessments (speed dating). To connect both needs, we tried to observe a trend of desires with regard to the things participants will really be excited about and willing to spend more time & money on.

We gathered our speed dating notes and went back to affinity diagramming boards, in order to formulate the most provocative concepts. Then we presented them to the class, the students were provoked.

 

Our top 3 contenders are:

Selfie Fairies

tp1
This idea tugs at attendees’ desire to document their experiences without stepping out of the experience itself. The ‘Selfie Fairies’, which are drones with cameras, would use UbiComp technology to identify important moments to take candid group photos during the music festival.

Music Decision Trees

tp2
This idea responds to the attendee desire of exploring and discovering new music. The festival schedule is set up so that lesser known acts perform during the day, while more mainstream acts perform at night. In this scenario the music decision trees would play tracks of the performers to help groups make decisions about their schedule.

Interactive/Collaborative Holograms

tp3
In this concept, we created an interactive hologram that captures attendees’ desire to interact more during the performance. This hologram appears at critical music moments, interacts with the surface area of the crowd and responds to attendee’s gestures.

 

Focusing the concept

From the three top scenarios, we decided to develop the interactive/collaborative holograms after presenting to the class & professor John Zimmerman.

[WHY NOT “Selfie Fairies”] While many of our participants found the ‘Selfie Fairies’ useful, their implementation is not unique to this music festival environment. [WHY NOT “Music Decision Trees”] “Music Decision Trees” helped attendees plan and navigate the event, but would only be a small part of their overall experience.

We want to design for the richest space, where we are able to deliver the design to a most desirable situation and help understand what’s happening in the concert by sensing people’s movements/energy, as well as compressing the time and space complexity for transformations using ubiquitous computing.

Our interviews told us people love large-scale collaborative experiences, but don’t want it to be an interfere of the artist performances. Therefore, we had to find a way to make the hologram not a gimmick to the audience, but co-created by the artist.

Technical feasibility

Our goal is to make sure that 90% of the technology is there to support our proposal. We’ll be using depth cameras to detect depth and see depth changes dynamically, so as to map specific gestures.

Another technical problem to look into is about the hologram that can represent a 3D view and be seen by large crowd. We researched online to see if existing technology can support our vision of a large-scale 3D volumetric hologram equation. And we found it is already possible to using projection mapping into fog stream and water droplets, and able to be interacted using motion parallax (http://www.es.osaka-u.ac.jp/en/research/news/20120427-182526.html).

At this point, with most parts of the technology able to support the design, we are more confident to move forward.

With regard to a 3D volumetric hologram equation, we also arranged a quick prototyping session to simulate the 3D volumetric display over fogs (we used baby powder instead), so that we can make sense if the direction of the technology can fit into our design solutions.

 

Video sketch script and the rough cut

Use video sketch method to help solidify the idea and its communication form

Even though we wanted to use location-based technology to provide more feedback of the hologram interactions, it is not very clear whether it appeals to the stakeholders who are going to invest and produce the new service, as it seems very expensive in making what we thought the infrared beam and readers on people’s bracelets would work. Moreover, are all examples we show effectively pitch that the interactive holograms are readily transferrable to all different types of high-energy concerts, styles of artists, and audience reactions? What crowd movements are representative enough to show?

Also, as we wanted to make the exclusive experience in the concerts extensible even after the events are finished, we need to figure out ways to retain that uniqueness for the concert goers, and also resinate with what the stakeholders (concert promoters) will want for successful concerts promotions for continuous years’ music festival.

We used video sketch to figure out those details and add to our designs. We created scripts, mapped with storyboards we had from previous concepts investigation, recorded our own voice, and then rough cuts of the video.

Decide rendering style and iteratively “sketch” the design

We finally decided the illustration style to be layered shots on top of the concert scenes. We can leverage stock photos that captured Coachella concert scenes. This helps us own the access to locations that would not be possible via traditional video and put things together faster with the goals in mind, while still communicating our interactive concert hologram concepts that are difficult to describe or prototype.

Pitching the rough cuts of the video sketch to the class, we prototyped ways that will help build the shared understanding about the design, so that it gives us further iteration ideas with respect to how to effectively help the stakeholders move forward.

 

Final presentation

We made the final pitch to the entire class, presenting the intelligent environment we designed. We showed process of how we reached the final design, and delivered the video sketch targeting at relevant stakeholders who will build this service.

The video sketch documented the user experience of the intelligent environment and explaining how the technology works that enables this experience.

Download our final presentation