Dynamic Labeling: A Control System for Labeling Styles in Image Annotation Tasks

Dynamic Labeling: A Control System for Labeling Styles in Image Annotation Tasks

Chia-Ming Chang Yi He Xusheng Du Xi Yang Haoran Xie

Abstract

Labeling style affects labeling efficiency and quality in image annotation tasks. For example, a “label quickly” style can increase labeling efficiency when the data are easy, and a “label carefully” style can increase label quali-ty when the data are difficult. However, the selection of an appropriate la-beling style is difficult as different annotators have different experiences and domain knowledge, affecting their subjective feelings of data difficul-ties (for example, User 1 feels Data A to be easy, while User 2 feels it diffi-cult). In this paper, we propose “Dynamic Labeling” as a control system for labeling styles used in image-labeling tasks. Our control system analyzes the labeling behaviors of annotators (i.e., label selection time) and dynami-cally assigns an appropriate labeling style (label quickly or label carefully). We conducted a user study to compare a conventional “non-dynamic” and the proposed “dynamic” labeling approaches for an image-labeling task. The results suggest that Dynamic Labeling increased the label quality and label-ing efficiency.

Video [3m28s]

Publication

Chia-Ming Chang, Yi He, Xusheng Du, Xi Yang and Haoran Xie. 2024, Dynamic Labeling: A Control System for Labeling Styles in Image Annotation Tasks. The 26th International Conference on Human-Computer Interaction (HCI International 2024), Washington DC, USA, 29 June - 4 July 2024. [PDF]

See more related works at "Labeling +" project.