Spatial Labeling: Leveraging Spatial Layout for Improving Label Quality in Non-Expert Image Annotation

Spatial Labeling: Leveraging Spatial Layout for Improving Label Quality in Non-Expert Image Annotation

Chia-Ming Chang Chia-Hsien Lee Takeo Igarashi

Abstract

Non-expert annotators (who lack sufficient domain knowledge) are often recruited for manual image labeling tasks owing to the lack of expert annotators. In such a case, label quality may be relatively low. We propose leveraging the spatial layout for improving label quality in non-expert image annotation. In the proposed system, an annotator first spatially lays out the incoming images and labels them on an open space, placing related items together. This serves as a working space (spatial organization) for tentative labeling. During the process, the annotator observes and organizes the similarities and differences between the items. Finally, the annotator provides definitive labels to the images based on the results of the spatial layout. We ran a user study comparing the proposed method and a traditional non-spatial layout in an image labeling task. The results demonstrated that annotators can complete the labeling tasks more accurately using the spatial layout interface than the non-spatial layout interface.

Video [2m53s]

Video preview [30s]

Presentation video [5m]

Publication

Chia-Ming Chang, Chia-Hsien Lee, and Takeo Igarashi. 2021. Spatial Labeling: Leveraging Spatial Layout for Improving Label Quality in Non-Expert Image Annotation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-12. DOI: https://doi.org/10.1145/3411764.3445165

Acknowledgement

This work was supported by (a) AIP Challenge Researcher (PI: Chia-Ming Chang) and (2) JST CREST (PI: Takeo Igarashi. Grant Number JPMJCR17A1, Japan). It is also an industry-academia collaborative project between UTokyo and LeadBest (Taiwan).

See more related works at "Labeling +" project.