Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array

Naoki Yoneoka, Yutaka Arakawa, Keiichi Yasumoto

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recently, smart speakers like Amazon Echo and Google Home have been spread widely. Those devices support users' life through voice interface by receiving voice commands to operate appliances and order goods to online shops. Meanwhile, it is reported that smart speakers are vulnerable to some malicious attacks which steal personal information and/or order unnecessary goods by uttering voice from a device nearby the speaker, abusing the fact that the smart speakers cannot distinguish human voice from machine voice. A new type of attack called DolphinAttack which utters ultrasonic voice inaudible to human is also reported. Therefore, a method to identify which of human or machine is sending voice commands to a smart speaker is desired. In this paper, to prevent such machine-voice based attacks to a smart speaker in absence of residents, we propose a system consisting of a speaker and a microphone array to detect the existence of a human nearby, supposing it can be incorporated in a smart speaker in the future. In our proposed system, the speaker emits sonar sound generated based on Orthogonal Frequency Division Multiplexing (OFDM) in all directions, the microphone array with 8 channels attached on top of the speaker receives the reflected sound, and the human existence is judged by comparing the reflected sound with that measured in the same environment without human. Through experiments with a prototype system, we confirmed that our proposed system can detect the human existence by measuring the reflected signal of 0.5 second.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages523-528
Number of pages6
ISBN (Electronic)9781538691519
DOIs
Publication statusPublished - Mar 1 2019
Event2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019 - Kyoto, Japan
Duration: Mar 11 2019Mar 15 2019

Publication series

Name2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019

Conference

Conference2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019
CountryJapan
CityKyoto
Period3/11/193/15/19

Fingerprint

Reverberation
Microphones
Acoustic waves
Sonar
Orthogonal frequency division multiplexing
Ultrasonics

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Cite this

Yoneoka, N., Arakawa, Y., & Yasumoto, K. (2019). Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019 (pp. 523-528). [8730674] (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/PERCOMW.2019.8730674

Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array. / Yoneoka, Naoki; Arakawa, Yutaka; Yasumoto, Keiichi.

2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 523-528 8730674 (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yoneoka, N, Arakawa, Y & Yasumoto, K 2019, Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array. in 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019., 8730674, 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019, Institute of Electrical and Electronics Engineers Inc., pp. 523-528, 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019, Kyoto, Japan, 3/11/19. https://doi.org/10.1109/PERCOMW.2019.8730674
Yoneoka N, Arakawa Y, Yasumoto K. Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 523-528. 8730674. (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019). https://doi.org/10.1109/PERCOMW.2019.8730674
Yoneoka, Naoki ; Arakawa, Yutaka ; Yasumoto, Keiichi. / Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array. 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 523-528 (2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019).
@inproceedings{4cc4fce494b34f8081ce29c2195c4fc1,
title = "Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array",
abstract = "Recently, smart speakers like Amazon Echo and Google Home have been spread widely. Those devices support users' life through voice interface by receiving voice commands to operate appliances and order goods to online shops. Meanwhile, it is reported that smart speakers are vulnerable to some malicious attacks which steal personal information and/or order unnecessary goods by uttering voice from a device nearby the speaker, abusing the fact that the smart speakers cannot distinguish human voice from machine voice. A new type of attack called DolphinAttack which utters ultrasonic voice inaudible to human is also reported. Therefore, a method to identify which of human or machine is sending voice commands to a smart speaker is desired. In this paper, to prevent such machine-voice based attacks to a smart speaker in absence of residents, we propose a system consisting of a speaker and a microphone array to detect the existence of a human nearby, supposing it can be incorporated in a smart speaker in the future. In our proposed system, the speaker emits sonar sound generated based on Orthogonal Frequency Division Multiplexing (OFDM) in all directions, the microphone array with 8 channels attached on top of the speaker receives the reflected sound, and the human existence is judged by comparing the reflected sound with that measured in the same environment without human. Through experiments with a prototype system, we confirmed that our proposed system can detect the human existence by measuring the reflected signal of 0.5 second.",
author = "Naoki Yoneoka and Yutaka Arakawa and Keiichi Yasumoto",
year = "2019",
month = "3",
day = "1",
doi = "10.1109/PERCOMW.2019.8730674",
language = "English",
series = "2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "523--528",
booktitle = "2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019",
address = "United States",

}

TY - GEN

T1 - Detecting Surrounding Users by Reverberation Analysis with a Smart Speaker and Microphone Array

AU - Yoneoka, Naoki

AU - Arakawa, Yutaka

AU - Yasumoto, Keiichi

PY - 2019/3/1

Y1 - 2019/3/1

N2 - Recently, smart speakers like Amazon Echo and Google Home have been spread widely. Those devices support users' life through voice interface by receiving voice commands to operate appliances and order goods to online shops. Meanwhile, it is reported that smart speakers are vulnerable to some malicious attacks which steal personal information and/or order unnecessary goods by uttering voice from a device nearby the speaker, abusing the fact that the smart speakers cannot distinguish human voice from machine voice. A new type of attack called DolphinAttack which utters ultrasonic voice inaudible to human is also reported. Therefore, a method to identify which of human or machine is sending voice commands to a smart speaker is desired. In this paper, to prevent such machine-voice based attacks to a smart speaker in absence of residents, we propose a system consisting of a speaker and a microphone array to detect the existence of a human nearby, supposing it can be incorporated in a smart speaker in the future. In our proposed system, the speaker emits sonar sound generated based on Orthogonal Frequency Division Multiplexing (OFDM) in all directions, the microphone array with 8 channels attached on top of the speaker receives the reflected sound, and the human existence is judged by comparing the reflected sound with that measured in the same environment without human. Through experiments with a prototype system, we confirmed that our proposed system can detect the human existence by measuring the reflected signal of 0.5 second.

AB - Recently, smart speakers like Amazon Echo and Google Home have been spread widely. Those devices support users' life through voice interface by receiving voice commands to operate appliances and order goods to online shops. Meanwhile, it is reported that smart speakers are vulnerable to some malicious attacks which steal personal information and/or order unnecessary goods by uttering voice from a device nearby the speaker, abusing the fact that the smart speakers cannot distinguish human voice from machine voice. A new type of attack called DolphinAttack which utters ultrasonic voice inaudible to human is also reported. Therefore, a method to identify which of human or machine is sending voice commands to a smart speaker is desired. In this paper, to prevent such machine-voice based attacks to a smart speaker in absence of residents, we propose a system consisting of a speaker and a microphone array to detect the existence of a human nearby, supposing it can be incorporated in a smart speaker in the future. In our proposed system, the speaker emits sonar sound generated based on Orthogonal Frequency Division Multiplexing (OFDM) in all directions, the microphone array with 8 channels attached on top of the speaker receives the reflected sound, and the human existence is judged by comparing the reflected sound with that measured in the same environment without human. Through experiments with a prototype system, we confirmed that our proposed system can detect the human existence by measuring the reflected signal of 0.5 second.

UR - http://www.scopus.com/inward/record.url?scp=85067971746&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067971746&partnerID=8YFLogxK

U2 - 10.1109/PERCOMW.2019.8730674

DO - 10.1109/PERCOMW.2019.8730674

M3 - Conference contribution

T3 - 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019

SP - 523

EP - 528

BT - 2019 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -