Abstract—Enabling interactive access to multimedia content and evaluating content-consumption behaviors and experiences involve several different research areas, which are covered at many different conferences. For four years, the Workshop on Interactive Content Consumption (WSICC) series offered a forum for combining interdisciplinary, comprehensive views, inspiring new discussions related to interactive multimedia. Here, the authors reflect on the outcome of the series.
Keywords—multimedia; video; Workshop on Interactive Content Consumption; WSICC; Internet/Web technologies; interactive content; hypermedia; human-computer interaction; content personalization; quality of experience; Virtual Director; distributed systems; internet /web technologies; visualization
The level of interactivity can vary when watching TV or online media. It can range from simple—where second-screen applications extend passive consumption with unsynchronized information about a program—to complex—where highly synchronized methods provide haptic feedback to actors of a remote live performance. Although simpler versions of interactive content enhancements are publicly available, more advanced methods and prototypes are still in the testing phase and thus stand to benefit from discourse in the community involving both researchers and practitioners.
Enabling interactive access to multimedia content and evaluating content-consumption behaviors and experiences involve several different research areas, which are covered at many different conferences. To enable a more focused discussion on interactive content consumption and its manifold factors and facets, during a project plenary meeting in Graz in 2012, the idea was born to conduct a related workshop at a scientific conference, which turned into the Workshop on Interactive Content Consumption (WSICC) series (https://wsicc.net).
The first WSICC was held in conjunction with the last European Conference on Interactive TV (euroITV) in 2013 (see Figure 1). EuroITV then became the ACM International Conference on Interactive Experiences for TV and Online Video (TVX), which is where the next three WSICC sessions were held (see Table 1). Over the years, we worked with different co-organizers, and the series concluded at the 2016 TVX conference (see Figure 2). In this article, we reflect on the outcome of the series.
Figure 1. Participants at the first Workshop on Interactive Content Consumption (WSICC) in 2013.
Figure 2. Participants at the last WSICC in Chicago in 2016.
New technologies, devices, media formats, and consumption paradigms keep emerging that allow for new types of interaction with media content. Such advances affect the user experience, its design, and scientific assessments. The timespan of the workshop series witnessed several innovations and trends, including high-resolution video, second-screen interaction, and audience feedback, all of which changed the complexion of the attention economy in content consumption and had implications on the user experience.
WSICC offered a forum for exploring various approaches to interactive multimedia content consumption, with contributions from not only research institutions but also industry leaders. With this mixed audience, we discussed recent technological advances and how they enabled new forms of content interaction—focusing on audiovisual media but also other modalities. We analyzed industry case studies, considering and evaluating user needs and the impact of technological advances. We also looked into more research-oriented, user-centric studies, which evaluated new types of audiovisual content interaction, especially in the realm of societal trends and media-consumption paradigm shifts.
The workshop’s scope remained fairly constant over the years and can be categorized along four dimensions: enabling technologies, interactive content, user experience, and user interaction.
The enabling technologies dimension researched content-adaptation techniques, including techniques for rendering and converting content, creating interactive personalization and recommendations, and promoting interactive and adaptive content delivery. It also explored studies on immersive devices, such as VR goggles, wearable devices, and cyber-physical systems. Furthermore, participants discussed novel approaches for producing content, inspiring interactive-media-coding technologies, and synchronizing and orchestrating media. They also reviewed tools for infusing interactivity into passive content.
The user experience dimension explored the quality of experience (QoE) theory and QoE evaluations, the impact and effects of interaction on perceived quality, the role of the audience, and the role of social context. It investigated the effect of increased interactivity, user engagement, and user empowerment, and it also reviewed user-overload and user-distraction evaluations and studies. Participants discussed consumption of personalized content, collaborative and community-based multimedia consumption and creation, immersive audiovisual content, and approaches for inclusion and improved accessibility.
The user interaction dimension analyzed novel interaction approaches, concepts, and paradigms. Participants discussed both computer-mediated communication scenarios and human-computer interaction aspects. Other topics included natural interaction techniques, multimodal interaction and social signal processing, social interaction during situated and mobile content consumption, feedback for user control, lean-forward interaction trends and joint interaction of larger groups, as well as the balance between active (lean-forward) and passive (lean-backward) content consumption.
The last dimension covered new forms of interactive content, such as content from gaming or from mobile, AR, and VR domains. Participants discussed both live and recorded materials, as well as data representation formats for interactive content, adaptable content, and content of variable length. They also debated high-quality and ultra-high-definition video content, sophisticated audio capture and playout, as well as content captured by novel types of sensors.
From the very beginning, the aim was to create a true workshop format in contrast to the standard sequence of oral talks with some subsequent Q&A, common at scientific conferences. One inspiration came from the Barcamp format, which puts a premium on interactive discussion and allows for effective knowledge transfer in a coffee-break-like atmosphere. In that realm, WSICC offered an opportunity for young students to present their ideas and get feedback at eye level from experienced members of the community.
We developed an interactive full-day format to stimulate both networking and knowledge transfer among participants. In an introduction round, everybody introduced themselves and revealed their research focus in the concise “3-hashtags” format, popular in Barcamps. Each workshop’s keynote gave a broad, interdisciplinary overview of the day’s focus area, and a set of talks provided detailed insights into recent research results—which also served as food for thought and a starting point for subsequent discourse. In principle, questions were also allowed during the talks.
The poster and demo session was kick-started with one-minute elevator pitches or a guided tour, and both formats served their purpose well. The highlight for many were two rounds of a “fishbowl” discussion, where seats were arranged in concentric circles. To actively contribute to the discussion, participants had to take one of the four or five seats in the innermost part of the circle (see Figure 3). This format enabled fruitful discussions among an ever-changing core group, while all other participants were close enough to listen in and quickly snatch an open seat whenever they felt they could contribute to the current topic.
Figure 3. The “fishbowl” format enabled fruitful discussions, with seats arranged in concentric circles. To actively contribute to the discussion, participants had to take one of the four or five seats in the innermost part of the circle. Participants engaged in fishbowl discussions in (a) Newcastle in 2014 and (b) Brussels in 2015, with the demos and posters in the background.
Hot topics and key discussion points during the talks and fishbowls were captured on sticky notes throughout the day and displayed on a large flipchart “mindmap.” Participants contributed to the mindmap as well (see Figure 4), again letting them provide input for further discussions. The four dimensions formed the backbones of these mindmap posters, capturing each workshop’s outcomes—that is, the main topics, trends, and open research questions.
Figure 4. Britta Meixner in front of the mindmap, with the four axes of the workshop, in Newcastle in 2014.
Every year, a keynote opened the WSICC workshop. All keynote speakers were well-known members of the research community, actively studying interactive multimedia systems. The first two talks in 2014 and 2015 focused more on systems, while the last two keynote talks in 2015 and 2016 focused more on QoE and the role of the user.
The first WSICC keynote speech was given by Wei Tsang Ooi, titled “The Best Interactive System is a Non-Interactive System.” The talk built on the hypothesis that when interaction can be exploited toward better system performance, the need for interaction will decrease. This hypothesis was supported by results from several studies, which were presented in a very descriptive manner.
In 2014, Marian Ursu’s keynote, “Blurring Boundaries: Intelligent Video Interaction,” looked at how film screen grammars can be applied in the domains of games, interactive storytelling systems, and video communication. It combined examples using recorded content with content based on live audio-visual content streams. This inspiring talk gave insight into several prototypes implemented in European research projects and their realization of interactive stories. In one, users could decide by vote how a story should continue. This approach was recently taken up by Netflix in the form of branching narratives.1
The 2015 WSICC keynote, “Enabling the ‘Wow!’ Multimedia Experience,” was held by Judith Redi and focused on multimedia delivery and QoE assessment research. Redi argued that users are no longer passive; rather, they are active in creating, interacting with, and selecting content. The talk outlined how to define the quality of a visual experience and discussed methods for experience assessment.
The most recent WSICC welcomed Louise Barkhuus as the keynote speaker. Her talk, “Interactive Live Performance Experiences,” shed light on recent technology advances for live performances that enable a new range of experiences as well as new forms of audience interaction and participation. As in Redi’s talk, the user’s QoE was a central point.
Looking back at the keynote speakers, we see a shift away from more systems- and content-oriented topics. The more recent focus seemed to be on the QoE in a new setting, which requires more or new interaction and can be difficult when first experienced. This shows a clear trend toward focusing on the user after solving the more technical issues.
In the four editions of this workshop, during each, three full papers were presented as talks. To encourage interaction between the participants, each workshop also had a very lively demo and poster session. A best paper award was given from 2013 to 2015, independent from the type of submitted paper.
In 2013, the award went to Chen Wang, Pablo Cesar, Erik Geelhoed, Ian Biscoe, and Phil Stenton, for “Sensing Audience Response—Beyond One Way Streaming of Live Performances.”2 The short paper reported on a field trial measuring and monitoring audience responses to performing arts. The authors used Galvanic Skin Response (GSR) sensors in an unusual approach based on “biofeedback.”
In 2014, the award went to Jeroen Vanattenhoven, David Geerts, and Dirk De Grooff, for “Television Experience Insights from HbbTV.”3 In this position paper, the authors sketched future research questions on the basis of previous studies. They briefly reported on an ethnographically inspired user study on second-screen usage and on a study comparing different interaction modalities. They further elaborated on experience measurement for group recommender systems.
In 2015, the award went to Lucia D’Acunto, Judith Redi, and Omar Niamut, for “iCaCoT—interactive Camera-based Coaching and Training.”4 This paper enabled interactive consumption of high-resolution video feeds via a tiled streaming approach, letting users exploit a simple, static, wide-angle camera feed and focus on specific areas, both spatially and temporally.
The interactive workshop format attracted many participants over the years. Some attended more than one edition of the workshop. In particular, the fishbowls let discussions adapt to the interests of the audience and highlight interesting topics that repeatedly came up during the day. Starting a fishbowl with one topic often led to different topics during the course of the discussion. Furthermore, each year, the mindmap was presented in the poster session of the main conference, resulting in additional discussions and creating an awareness of the workshop by advertising its results to a bigger audience. You can find the mindmaps as photos linked on the workshop’s website.
All authors from 2013 to 2015 were invited to submit their research to the Multimedia Tools and Applications Journal’s special issue, titled “Interactive Media: Technology and Experience.”5 This special issue gave authors the opportunity to present further research and more comprehensive studies. The special issue, published in 2017, concluded the workshop series.
The workshop series’ success proves that the topic of interactive content consumption remains a relevant research area that integrates several disciplines. The 2017 edition of TVX showed several demos, presentations, and workshops in this area, illustrating the topicality and importance of the workshop topics. In that sense, the “In-Programme Personalisation for Broadcast” workshop or the “Workshop on Interactive Digital Storytelling in Broadcasting” at TVX 2017, or the “Multimedia Alternate Realities” workshop at ACM Multimedia later this year, can be regarded as a more focused continuation of WSICC’s broader scope.
New technologies, devices, and content formats keep emerging, requiring more research to understand their potential and how they affect users’ needs, interactions, and experiences. This is especially true for VR and AR applications, as well as for other applications that move beyond traditional sound and vision. The industry must find device and application scenarios that allow for meaningful use of these technologies without overwhelming users with unknown types of input. Users in turn must then learn how to interact in such settings—with the content and with each other in remote settings.
For example, at WSICC 2013, researchers presented impressive advancements in gesture-based interaction, but this type of interaction hadn’t yet entered our everyday lives as many had anticipated. On the other hand, during WSICC 2015, we saw an innovation targeting a well-defined application domain with the Smart Ski Goggles, showcased by Gerald Binder, which went on to become a successful product. We saw similar developments with other topics, such as hypermedia, Virtual Director technology, live video communication, and second-screen interaction, during the WSICC lifetime. Whether or not the recent surge in 360-degree video, vertical video, and AR/VR content types will shift back the community’s focus to new enabling technologies remains to be seen.
Scientific discourse at conferences typically focuses on a single perspective, reflecting the conference’s main topic. For four years, WSICC offered a forum for combining interdisciplinary, comprehensive views, inspiring new discussions. Although the series has come to an end, related research and discussions will continue, driven by the workshop participants and other researchers in the community.
A key takeaway from the series, based on feedback from many participants, is that researchers enjoy interactive workshop formats at scientific conferences. Somewhat unexpectedly, the fishbowl format became a trademark for WSICC. Participants generally embraced it, saying it was engaging while letting everybody weigh in. We encourage future workshop organizers to experiment with interactive formats by adapting WSICC’s style or by experimenting with other elements. Keep in mind that intense discourse can be quite exhausting, so it’s important to balance the sequence of sessions within the workshop.
We want to thank all of the WSICC co-organizers, reviewers, conference organizers, authors, keynote speakers, and participants who contributed to this inspiring workshop series.