Publication detail
Implicit Hand Gestures in Aeronautics Cockpit as a Cue for Crew State and Workload Inference
BEHÚŇ, K. HEROUT, A. PAVELKOVÁ, A.
Original Title
Implicit Hand Gestures in Aeronautics Cockpit as a Cue for Crew State and Workload Inference
English Title
Implicit Hand Gestures in Aeronautics Cockpit as a Cue for Crew State and Workload Inference
Type
conference paper
Language
en
Original Abstract
This paper aims at improving advanced aeronautic cockpit by raising its awareness of the crew's state and workload level. Our approach is based on visual analysis of pilot's upper body movements. We define the term of "implicit gestures" and further observe its subclasses. We collected a simulator dataset of practical implicit gestures, annotated semi-automatically a dataset for Human pose estimation training, and we offer these datasets for public use. Based on experiments on this data, we propose a method for recognition of implicit gestures - full interactions, touch-and-go interactions, and unfinished gestures. Our approach is purely visual (no depth data, which are hardly usable in the cockpit environment due to regulations). This method is based on human pose estimation by a hierarchical approach named Pose machine whose subsampled output is used for recognition of implicit gesture presence from sequences of frames by random forest. The experiments show that the classification works reliably and the method is able to recognize these implicit gestures in the cockpit.
English abstract
This paper aims at improving advanced aeronautic cockpit by raising its awareness of the crew's state and workload level. Our approach is based on visual analysis of pilot's upper body movements. We define the term of "implicit gestures" and further observe its subclasses. We collected a simulator dataset of practical implicit gestures, annotated semi-automatically a dataset for Human pose estimation training, and we offer these datasets for public use. Based on experiments on this data, we propose a method for recognition of implicit gestures - full interactions, touch-and-go interactions, and unfinished gestures. Our approach is purely visual (no depth data, which are hardly usable in the cockpit environment due to regulations). This method is based on human pose estimation by a hierarchical approach named Pose machine whose subsampled output is used for recognition of implicit gesture presence from sequences of frames by random forest. The experiments show that the classification works reliably and the method is able to recognize these implicit gestures in the cockpit.
Keywords
Implicit Gestures, Human Pose Estimation, Random Forest, Aeronautics Cockpit, Visual Recognition
RIV year
2015
Released
01.07.2015
Publisher
The Universidad de Las Palmas de Gran Canaria
Location
Las Palmas
ISBN
978-1-4673-6596-3
Book
Proceedings of ITSC 2015
Edition
NEUVEDEN
Edition number
NEUVEDEN
Pages from
632
Pages to
637
Pages count
6
Documents
BibTex
@inproceedings{BUT119871,
author="Kamil {Behúň} and Adam {Herout} and Alena {Pavelková}",
title="Implicit Hand Gestures in Aeronautics Cockpit as a Cue for Crew State and Workload Inference",
annote="This paper aims at improving advanced aeronautic cockpit by raising its awareness
of the crew's state and workload level. Our approach is based on visual analysis
of pilot's upper body movements. We define the term of "implicit gestures" and
further observe its subclasses. We collected a simulator dataset of practical
implicit gestures, annotated semi-automatically a dataset for Human pose
estimation training, and we offer these datasets for public use. Based on
experiments on this data, we propose a method for recognition of implicit
gestures - full interactions, touch-and-go interactions, and unfinished gestures.
Our approach is purely visual (no depth data, which are hardly usable in the
cockpit environment due to regulations). This method is based on human pose
estimation by a hierarchical approach named Pose machine whose subsampled output
is used for recognition of implicit gesture presence from sequences of frames by
random forest. The experiments show that the classification works reliably and
the method is able to recognize these implicit gestures in the cockpit.",
address="The Universidad de Las Palmas de Gran Canaria",
booktitle="Proceedings of ITSC 2015",
chapter="119871",
doi="10.1109/ITSC.2015.109",
edition="NEUVEDEN",
howpublished="online",
institution="The Universidad de Las Palmas de Gran Canaria",
year="2015",
month="july",
pages="632--637",
publisher="The Universidad de Las Palmas de Gran Canaria",
type="conference paper"
}