Publication detail

Impact of Test Condition Selection in Adaptive Crowdsourcing Studies on Subjective Quality

SEUFERT, M. ZACH, O. HOSSFELD, T. SLANINA, M. TRAN-GIA, P.

Original Title

Impact of Test Condition Selection in Adaptive Crowdsourcing Studies on Subjective Quality

English Title

Impact of Test Condition Selection in Adaptive Crowdsourcing Studies on Subjective Quality

Type

conference paper

Language

en

Original Abstract

Adaptive crowdsourcing is a new approach to crowdsourced Quality of Experience (QoE) studies, which aims to improve the certainty of resulting QoE models by adaptively distributing a fixed budget of user ratings to the test conditions. The main idea of the adaptation is to dynamically allocate the next rating to a condition, for which the submitted ratings so far show a low certainty. This paper investigates the effects of statistical adaptation on the distribution of ratings and the goodness of the resulting QoE models. Thereby, it gives methodological advice how to select test conditions for future crowdsourced QoE studies.

English abstract

Adaptive crowdsourcing is a new approach to crowdsourced Quality of Experience (QoE) studies, which aims to improve the certainty of resulting QoE models by adaptively distributing a fixed budget of user ratings to the test conditions. The main idea of the adaptation is to dynamically allocate the next rating to a condition, for which the submitted ratings so far show a low certainty. This paper investigates the effects of statistical adaptation on the distribution of ratings and the goodness of the resulting QoE models. Thereby, it gives methodological advice how to select test conditions for future crowdsourced QoE studies.

Keywords

QoE, test condiition, crowdsourcing, adaptive test design

Released

06.06.2016

ISBN

978-1-5090-0354-9

Book

Proceedings of 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX)

Pages from

1

Pages to

6

Pages count

1

BibTex


@inproceedings{BUT126583,
  author="Michael {Seufert} and Ondřej {Zach} and Tobias {Hossfeld} and Martin {Slanina} and Phuoc {Tran-Gia}",
  title="Impact of Test Condition Selection in Adaptive Crowdsourcing Studies on Subjective Quality",
  annote="Adaptive crowdsourcing is a new approach to
crowdsourced Quality of Experience (QoE) studies, which aims
to improve the certainty of resulting QoE models by adaptively
distributing a fixed budget of user ratings to the test conditions.
The main idea of the adaptation is to dynamically allocate the next
rating to a condition, for which the submitted ratings so far show
a low certainty. This paper investigates the effects of statistical
adaptation on the distribution of ratings and the goodness of the
resulting QoE models. Thereby, it gives methodological advice
how to select test conditions for future crowdsourced QoE studies.",
  booktitle="Proceedings of 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX)",
  chapter="126583",
  doi="10.1109/QoMEX.2016.7498939",
  howpublished="electronic, physical medium",
  year="2016",
  month="june",
  pages="1--6",
  type="conference paper"
}