Intelligent biohazard training based on real-time task recognition

Helmut Prendinger, Nahum Alvarez, Antonio Sanchez-Ruiz, Marc Cavazza, João Catarino, João Oliveira, Rui Prada, Shuji Fujimoto, Mika Shigematsu

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Virtual environments offer an ideal setting to develop intelligent training applications. Yet, their ability to support complex procedures depends on the appropriate integration of knowledge-based techniques and natural interaction. In this article, we describe the implementation of an intelligent rehearsal system for biohazard laboratory procedures, based on the real-time instantiation of task models from the trainee's actions A virtual biohazard laboratory has been recreated using the Unity3D engine, in which users interact with laboratory objects using keyboard/mouse input or hand gestures through a Kinect device. Realistic behavior for objects is supported by the implementation of a relevant subset of common sense and physics knowledge. User interaction with objects leads to the recognition of specific actions, which are used to progressively instantiate a task-based representation of biohazard procedures. The dynamics of this instantiation process supports trainee evaluation as well as real-time assistance. This system is designed primarily as a rehearsa system providing real-time advice and supporting user performance evaluation. We provide detailed examples illustrating error detection and recovery, and results from on-site testing with students from the Faculty of Medical Sciences at Kyushu University. In the study, we investigate the usability aspect by comparing interaction with mouse and Kinect devices and the effect of real-time task recognition on recovery time after user mistakes.

Original languageEnglish
Article number21
JournalACM Transactions on Interactive Intelligent Systems
Volume6
Issue number3
DOIs
Publication statusPublished - Sep 2016

Fingerprint

Biohazards
Error detection
Intelligent systems
Real time systems
Virtual reality
Physics
Students
Engines
Recovery
Testing

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Artificial Intelligence

Cite this

Prendinger, H., Alvarez, N., Sanchez-Ruiz, A., Cavazza, M., Catarino, J., Oliveira, J., ... Shigematsu, M. (2016). Intelligent biohazard training based on real-time task recognition. ACM Transactions on Interactive Intelligent Systems, 6(3), [21]. https://doi.org/10.1145/2883617

Intelligent biohazard training based on real-time task recognition. / Prendinger, Helmut; Alvarez, Nahum; Sanchez-Ruiz, Antonio; Cavazza, Marc; Catarino, João; Oliveira, João; Prada, Rui; Fujimoto, Shuji; Shigematsu, Mika.

In: ACM Transactions on Interactive Intelligent Systems, Vol. 6, No. 3, 21, 09.2016.

Research output: Contribution to journalArticle

Prendinger, H, Alvarez, N, Sanchez-Ruiz, A, Cavazza, M, Catarino, J, Oliveira, J, Prada, R, Fujimoto, S & Shigematsu, M 2016, 'Intelligent biohazard training based on real-time task recognition', ACM Transactions on Interactive Intelligent Systems, vol. 6, no. 3, 21. https://doi.org/10.1145/2883617
Prendinger H, Alvarez N, Sanchez-Ruiz A, Cavazza M, Catarino J, Oliveira J et al. Intelligent biohazard training based on real-time task recognition. ACM Transactions on Interactive Intelligent Systems. 2016 Sep;6(3). 21. https://doi.org/10.1145/2883617
Prendinger, Helmut ; Alvarez, Nahum ; Sanchez-Ruiz, Antonio ; Cavazza, Marc ; Catarino, João ; Oliveira, João ; Prada, Rui ; Fujimoto, Shuji ; Shigematsu, Mika. / Intelligent biohazard training based on real-time task recognition. In: ACM Transactions on Interactive Intelligent Systems. 2016 ; Vol. 6, No. 3.
@article{0e47c1c6248e46689a8084ed53953dac,
title = "Intelligent biohazard training based on real-time task recognition",
abstract = "Virtual environments offer an ideal setting to develop intelligent training applications. Yet, their ability to support complex procedures depends on the appropriate integration of knowledge-based techniques and natural interaction. In this article, we describe the implementation of an intelligent rehearsal system for biohazard laboratory procedures, based on the real-time instantiation of task models from the trainee's actions A virtual biohazard laboratory has been recreated using the Unity3D engine, in which users interact with laboratory objects using keyboard/mouse input or hand gestures through a Kinect device. Realistic behavior for objects is supported by the implementation of a relevant subset of common sense and physics knowledge. User interaction with objects leads to the recognition of specific actions, which are used to progressively instantiate a task-based representation of biohazard procedures. The dynamics of this instantiation process supports trainee evaluation as well as real-time assistance. This system is designed primarily as a rehearsa system providing real-time advice and supporting user performance evaluation. We provide detailed examples illustrating error detection and recovery, and results from on-site testing with students from the Faculty of Medical Sciences at Kyushu University. In the study, we investigate the usability aspect by comparing interaction with mouse and Kinect devices and the effect of real-time task recognition on recovery time after user mistakes.",
author = "Helmut Prendinger and Nahum Alvarez and Antonio Sanchez-Ruiz and Marc Cavazza and Jo{\~a}o Catarino and Jo{\~a}o Oliveira and Rui Prada and Shuji Fujimoto and Mika Shigematsu",
year = "2016",
month = "9",
doi = "10.1145/2883617",
language = "English",
volume = "6",
journal = "ACM Transactions on Interactive Intelligent Systems",
issn = "2160-6455",
publisher = "Association for Computing Machinery (ACM)",
number = "3",

}

TY - JOUR

T1 - Intelligent biohazard training based on real-time task recognition

AU - Prendinger, Helmut

AU - Alvarez, Nahum

AU - Sanchez-Ruiz, Antonio

AU - Cavazza, Marc

AU - Catarino, João

AU - Oliveira, João

AU - Prada, Rui

AU - Fujimoto, Shuji

AU - Shigematsu, Mika

PY - 2016/9

Y1 - 2016/9

N2 - Virtual environments offer an ideal setting to develop intelligent training applications. Yet, their ability to support complex procedures depends on the appropriate integration of knowledge-based techniques and natural interaction. In this article, we describe the implementation of an intelligent rehearsal system for biohazard laboratory procedures, based on the real-time instantiation of task models from the trainee's actions A virtual biohazard laboratory has been recreated using the Unity3D engine, in which users interact with laboratory objects using keyboard/mouse input or hand gestures through a Kinect device. Realistic behavior for objects is supported by the implementation of a relevant subset of common sense and physics knowledge. User interaction with objects leads to the recognition of specific actions, which are used to progressively instantiate a task-based representation of biohazard procedures. The dynamics of this instantiation process supports trainee evaluation as well as real-time assistance. This system is designed primarily as a rehearsa system providing real-time advice and supporting user performance evaluation. We provide detailed examples illustrating error detection and recovery, and results from on-site testing with students from the Faculty of Medical Sciences at Kyushu University. In the study, we investigate the usability aspect by comparing interaction with mouse and Kinect devices and the effect of real-time task recognition on recovery time after user mistakes.

AB - Virtual environments offer an ideal setting to develop intelligent training applications. Yet, their ability to support complex procedures depends on the appropriate integration of knowledge-based techniques and natural interaction. In this article, we describe the implementation of an intelligent rehearsal system for biohazard laboratory procedures, based on the real-time instantiation of task models from the trainee's actions A virtual biohazard laboratory has been recreated using the Unity3D engine, in which users interact with laboratory objects using keyboard/mouse input or hand gestures through a Kinect device. Realistic behavior for objects is supported by the implementation of a relevant subset of common sense and physics knowledge. User interaction with objects leads to the recognition of specific actions, which are used to progressively instantiate a task-based representation of biohazard procedures. The dynamics of this instantiation process supports trainee evaluation as well as real-time assistance. This system is designed primarily as a rehearsa system providing real-time advice and supporting user performance evaluation. We provide detailed examples illustrating error detection and recovery, and results from on-site testing with students from the Faculty of Medical Sciences at Kyushu University. In the study, we investigate the usability aspect by comparing interaction with mouse and Kinect devices and the effect of real-time task recognition on recovery time after user mistakes.

UR - http://www.scopus.com/inward/record.url?scp=84989807484&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84989807484&partnerID=8YFLogxK

U2 - 10.1145/2883617

DO - 10.1145/2883617

M3 - Article

AN - SCOPUS:84989807484

VL - 6

JO - ACM Transactions on Interactive Intelligent Systems

JF - ACM Transactions on Interactive Intelligent Systems

SN - 2160-6455

IS - 3

M1 - 21

ER -