The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions

Victoria Korshunova, Gerard Bastiaan Remijn, Synes Elischka, Catarina Mendonca

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With fast technological developments, traditional perceptual environments disappear and new ones emerge. These changes make the human senses adapt to new ways of perceptual understanding, for example, regarding the perceptual integration of sound and vision. Proceeding from the fact that hearing cooperates with visual attention processes, the aim of this study is to investigate the effect of different sound design conditions on the perception of cinematic content in immersive audiovisual reproductions. Here we introduce the results of a visual selective attention task (counting objects) performed by participants watching a 270-degree immersive audiovisual display, on which a movie (»Ego Cure») was shown. Four sound conditions were used, which employed an increasing number of loudspeakers, i.e., mono, stereo, 5.1 and 7.1.4. Eye tracking was used to record the participant's eye gaze during the task. The eye tracking data showed that an increased number of speakers and a wider spatial audio distribution diffused the participants' attention from the task-related part of the display to non-task-related directions. The number of participants looking at the task-irrelevant display in the 7.1.4 condition was significantly higher than in the mono audio condition. This implies that additional spatial cues in the auditory modality automatically influence human visual attention (involuntary eye movements) and human analysis of visual information. Sound engineers should consider this when mixing educational or any other information-oriented productions.

Original languageEnglish
Title of host publicationProceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
EditorsDongdong Weng, Liwei Chan, Youngho Lee, Xiaohui Liang, Nobuchika Sakata
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728115719
DOIs
Publication statusPublished - May 7 2019
Event12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 - Ikoma, Nara, Japan
Duration: Mar 28 2019Mar 29 2019

Publication series

NameProceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

Conference

Conference12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
CountryJapan
CityIkoma, Nara
Period3/28/193/29/19

Fingerprint

Acoustic waves
Display devices
sound engineer
Loudspeakers
Ego
Eye movements
Dyskinesias
Audition
Motion Pictures
Eye Movements
technical development
movies
Hearing
Cues
Reproduction
Engineers
Visual attention
Technological development
Education
Movies

All Science Journal Classification (ASJC) codes

  • Education
  • Sociology and Political Science
  • Communication
  • Marketing
  • Strategy and Management
  • Computer Networks and Communications
  • Information Systems and Management
  • Social Psychology

Cite this

Korshunova, V., Remijn, G. B., Elischka, S., & Mendonca, C. (2019). The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions. In D. Weng, L. Chan, Y. Lee, X. Liang, & N. Sakata (Eds.), Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 [8709163] (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/APMAR.2019.8709163

The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions. / Korshunova, Victoria; Remijn, Gerard Bastiaan; Elischka, Synes; Mendonca, Catarina.

Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. ed. / Dongdong Weng; Liwei Chan; Youngho Lee; Xiaohui Liang; Nobuchika Sakata. Institute of Electrical and Electronics Engineers Inc., 2019. 8709163 (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Korshunova, V, Remijn, GB, Elischka, S & Mendonca, C 2019, The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions. in D Weng, L Chan, Y Lee, X Liang & N Sakata (eds), Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019., 8709163, Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019, Institute of Electrical and Electronics Engineers Inc., 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019, Ikoma, Nara, Japan, 3/28/19. https://doi.org/10.1109/APMAR.2019.8709163
Korshunova V, Remijn GB, Elischka S, Mendonca C. The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions. In Weng D, Chan L, Lee Y, Liang X, Sakata N, editors, Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8709163. (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019). https://doi.org/10.1109/APMAR.2019.8709163
Korshunova, Victoria ; Remijn, Gerard Bastiaan ; Elischka, Synes ; Mendonca, Catarina. / The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions. Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. editor / Dongdong Weng ; Liwei Chan ; Youngho Lee ; Xiaohui Liang ; Nobuchika Sakata. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019).
@inproceedings{5d8510f846b3407fb5a0f3d9d10cb40c,
title = "The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions",
abstract = "With fast technological developments, traditional perceptual environments disappear and new ones emerge. These changes make the human senses adapt to new ways of perceptual understanding, for example, regarding the perceptual integration of sound and vision. Proceeding from the fact that hearing cooperates with visual attention processes, the aim of this study is to investigate the effect of different sound design conditions on the perception of cinematic content in immersive audiovisual reproductions. Here we introduce the results of a visual selective attention task (counting objects) performed by participants watching a 270-degree immersive audiovisual display, on which a movie (»Ego Cure») was shown. Four sound conditions were used, which employed an increasing number of loudspeakers, i.e., mono, stereo, 5.1 and 7.1.4. Eye tracking was used to record the participant's eye gaze during the task. The eye tracking data showed that an increased number of speakers and a wider spatial audio distribution diffused the participants' attention from the task-related part of the display to non-task-related directions. The number of participants looking at the task-irrelevant display in the 7.1.4 condition was significantly higher than in the mono audio condition. This implies that additional spatial cues in the auditory modality automatically influence human visual attention (involuntary eye movements) and human analysis of visual information. Sound engineers should consider this when mixing educational or any other information-oriented productions.",
author = "Victoria Korshunova and Remijn, {Gerard Bastiaan} and Synes Elischka and Catarina Mendonca",
year = "2019",
month = "5",
day = "7",
doi = "10.1109/APMAR.2019.8709163",
language = "English",
series = "Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
editor = "Dongdong Weng and Liwei Chan and Youngho Lee and Xiaohui Liang and Nobuchika Sakata",
booktitle = "Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019",
address = "United States",

}

TY - GEN

T1 - The Impact of Sound Systems on the Perception of Cinematic Content in Immersive Audiovisual Productions

AU - Korshunova, Victoria

AU - Remijn, Gerard Bastiaan

AU - Elischka, Synes

AU - Mendonca, Catarina

PY - 2019/5/7

Y1 - 2019/5/7

N2 - With fast technological developments, traditional perceptual environments disappear and new ones emerge. These changes make the human senses adapt to new ways of perceptual understanding, for example, regarding the perceptual integration of sound and vision. Proceeding from the fact that hearing cooperates with visual attention processes, the aim of this study is to investigate the effect of different sound design conditions on the perception of cinematic content in immersive audiovisual reproductions. Here we introduce the results of a visual selective attention task (counting objects) performed by participants watching a 270-degree immersive audiovisual display, on which a movie (»Ego Cure») was shown. Four sound conditions were used, which employed an increasing number of loudspeakers, i.e., mono, stereo, 5.1 and 7.1.4. Eye tracking was used to record the participant's eye gaze during the task. The eye tracking data showed that an increased number of speakers and a wider spatial audio distribution diffused the participants' attention from the task-related part of the display to non-task-related directions. The number of participants looking at the task-irrelevant display in the 7.1.4 condition was significantly higher than in the mono audio condition. This implies that additional spatial cues in the auditory modality automatically influence human visual attention (involuntary eye movements) and human analysis of visual information. Sound engineers should consider this when mixing educational or any other information-oriented productions.

AB - With fast technological developments, traditional perceptual environments disappear and new ones emerge. These changes make the human senses adapt to new ways of perceptual understanding, for example, regarding the perceptual integration of sound and vision. Proceeding from the fact that hearing cooperates with visual attention processes, the aim of this study is to investigate the effect of different sound design conditions on the perception of cinematic content in immersive audiovisual reproductions. Here we introduce the results of a visual selective attention task (counting objects) performed by participants watching a 270-degree immersive audiovisual display, on which a movie (»Ego Cure») was shown. Four sound conditions were used, which employed an increasing number of loudspeakers, i.e., mono, stereo, 5.1 and 7.1.4. Eye tracking was used to record the participant's eye gaze during the task. The eye tracking data showed that an increased number of speakers and a wider spatial audio distribution diffused the participants' attention from the task-related part of the display to non-task-related directions. The number of participants looking at the task-irrelevant display in the 7.1.4 condition was significantly higher than in the mono audio condition. This implies that additional spatial cues in the auditory modality automatically influence human visual attention (involuntary eye movements) and human analysis of visual information. Sound engineers should consider this when mixing educational or any other information-oriented productions.

UR - http://www.scopus.com/inward/record.url?scp=85066332801&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066332801&partnerID=8YFLogxK

U2 - 10.1109/APMAR.2019.8709163

DO - 10.1109/APMAR.2019.8709163

M3 - Conference contribution

AN - SCOPUS:85066332801

T3 - Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

BT - Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

A2 - Weng, Dongdong

A2 - Chan, Liwei

A2 - Lee, Youngho

A2 - Liang, Xiaohui

A2 - Sakata, Nobuchika

PB - Institute of Electrical and Electronics Engineers Inc.

ER -