Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices

Dmitrii Fedotov, Yuki Matsuda, Yuta Takahashi, Yutaka Arakawa, Keiichi Yasumoto, Wolfgang Minker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Following the technical progress and growing touristic market, demand on guidance systems is constantly increasing. Current systems are not personalized, they usually provide only a general information on sightseeing spot and do not concern about the tourist's perception of it. To design more adjustable and context-aware system, we focus on collecting and estimating emotions and satisfaction level, those tourists experience during the sightseeing tour. We reducing changes in their behaviour by collecting two types of information: conscious (short videos with impressions) and unconscious (behavioural pattern recorded with wearable devices) continuously during the whole tour. We have conducted experiments and collected initial data to build the prototype system. For each sight of the tour, participants provided an emotion and satisfaction labels. We use them to train unimodal neural network based models, fuse them together and get the final prediction for each recording. As tourist himself is the only source of labels for such system, we introduce an approach of post-experimental label correction, based on paired comparison. Such system built together allows us to use different modalities or their combination to perform real-time tourist emotion recognition and satisfaction estimation in-the-wild, bringing touristic guidance systems to the new level.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages358-360
Number of pages3
ISBN (Electronic)9781538691519
DOIs
Publication statusPublished - Mar 1 2019
Event2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019 - Kyoto, Japan
Duration: Mar 11 2019Mar 15 2019

Publication series

Name2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019

Conference

Conference2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019
CountryJapan
CityKyoto
Period3/11/193/15/19

Fingerprint

Labels
Electric fuses
Neural networks
Emotion
Experiments
Tourists

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Cite this

Fedotov, D., Matsuda, Y., Takahashi, Y., Arakawa, Y., Yasumoto, K., & Minker, W. (2019). Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019 (pp. 358-360). [8730864] (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/PERCOMW.2019.8730864

Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices. / Fedotov, Dmitrii; Matsuda, Yuki; Takahashi, Yuta; Arakawa, Yutaka; Yasumoto, Keiichi; Minker, Wolfgang.

2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 358-360 8730864 (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fedotov, D, Matsuda, Y, Takahashi, Y, Arakawa, Y, Yasumoto, K & Minker, W 2019, Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices. in 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019., 8730864, 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019, Institute of Electrical and Electronics Engineers Inc., pp. 358-360, 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019, Kyoto, Japan, 3/11/19. https://doi.org/10.1109/PERCOMW.2019.8730864
Fedotov D, Matsuda Y, Takahashi Y, Arakawa Y, Yasumoto K, Minker W. Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 358-360. 8730864. (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019). https://doi.org/10.1109/PERCOMW.2019.8730864
Fedotov, Dmitrii ; Matsuda, Yuki ; Takahashi, Yuta ; Arakawa, Yutaka ; Yasumoto, Keiichi ; Minker, Wolfgang. / Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices. 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 358-360 (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019).
@inproceedings{9dcb86bdee9941ac933efbec502d1936,
title = "Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices",
abstract = "Following the technical progress and growing touristic market, demand on guidance systems is constantly increasing. Current systems are not personalized, they usually provide only a general information on sightseeing spot and do not concern about the tourist's perception of it. To design more adjustable and context-aware system, we focus on collecting and estimating emotions and satisfaction level, those tourists experience during the sightseeing tour. We reducing changes in their behaviour by collecting two types of information: conscious (short videos with impressions) and unconscious (behavioural pattern recorded with wearable devices) continuously during the whole tour. We have conducted experiments and collected initial data to build the prototype system. For each sight of the tour, participants provided an emotion and satisfaction labels. We use them to train unimodal neural network based models, fuse them together and get the final prediction for each recording. As tourist himself is the only source of labels for such system, we introduce an approach of post-experimental label correction, based on paired comparison. Such system built together allows us to use different modalities or their combination to perform real-time tourist emotion recognition and satisfaction estimation in-the-wild, bringing touristic guidance systems to the new level.",
author = "Dmitrii Fedotov and Yuki Matsuda and Yuta Takahashi and Yutaka Arakawa and Keiichi Yasumoto and Wolfgang Minker",
year = "2019",
month = "3",
day = "1",
doi = "10.1109/PERCOMW.2019.8730864",
language = "English",
series = "2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "358--360",
booktitle = "2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019",
address = "United States",

}

TY - GEN

T1 - Towards Real-Time Contextual Touristic Emotion and Satisfaction Estimation with Wearable Devices

AU - Fedotov, Dmitrii

AU - Matsuda, Yuki

AU - Takahashi, Yuta

AU - Arakawa, Yutaka

AU - Yasumoto, Keiichi

AU - Minker, Wolfgang

PY - 2019/3/1

Y1 - 2019/3/1

N2 - Following the technical progress and growing touristic market, demand on guidance systems is constantly increasing. Current systems are not personalized, they usually provide only a general information on sightseeing spot and do not concern about the tourist's perception of it. To design more adjustable and context-aware system, we focus on collecting and estimating emotions and satisfaction level, those tourists experience during the sightseeing tour. We reducing changes in their behaviour by collecting two types of information: conscious (short videos with impressions) and unconscious (behavioural pattern recorded with wearable devices) continuously during the whole tour. We have conducted experiments and collected initial data to build the prototype system. For each sight of the tour, participants provided an emotion and satisfaction labels. We use them to train unimodal neural network based models, fuse them together and get the final prediction for each recording. As tourist himself is the only source of labels for such system, we introduce an approach of post-experimental label correction, based on paired comparison. Such system built together allows us to use different modalities or their combination to perform real-time tourist emotion recognition and satisfaction estimation in-the-wild, bringing touristic guidance systems to the new level.

AB - Following the technical progress and growing touristic market, demand on guidance systems is constantly increasing. Current systems are not personalized, they usually provide only a general information on sightseeing spot and do not concern about the tourist's perception of it. To design more adjustable and context-aware system, we focus on collecting and estimating emotions and satisfaction level, those tourists experience during the sightseeing tour. We reducing changes in their behaviour by collecting two types of information: conscious (short videos with impressions) and unconscious (behavioural pattern recorded with wearable devices) continuously during the whole tour. We have conducted experiments and collected initial data to build the prototype system. For each sight of the tour, participants provided an emotion and satisfaction labels. We use them to train unimodal neural network based models, fuse them together and get the final prediction for each recording. As tourist himself is the only source of labels for such system, we introduce an approach of post-experimental label correction, based on paired comparison. Such system built together allows us to use different modalities or their combination to perform real-time tourist emotion recognition and satisfaction estimation in-the-wild, bringing touristic guidance systems to the new level.

UR - http://www.scopus.com/inward/record.url?scp=85067921505&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067921505&partnerID=8YFLogxK

U2 - 10.1109/PERCOMW.2019.8730864

DO - 10.1109/PERCOMW.2019.8730864

M3 - Conference contribution

T3 - 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019

SP - 358

EP - 360

BT - 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -