Interactive Visual Labelling versus Active Learning: An Experimental Comparison

Mohammad Chegini, Jürgen Bernard, Jian Cui, Fatemeh Chegini, Alexei Sourin, Keith Andrews, Tobias Schreck

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Methods from supervised machine learning allow the classification of
new data automatically and are tremendously helpful for data analysis.
The quality of supervised learning depends not only on the type of
algorithm used but, importantly, also on the quality of the labelled
dataset used to train the classifier. Labelling instances in a
training dataset is often done manually, relying on selections and
annotations by expert analysts and is often a tedious and
time-consuming process.

Active learning algorithms can automatically determine a subset of
data instances for which labels would provide useful input to the
learning process. Interactive visual labelling techniques are a
promising alternative, providing effective visual overviews from which
an analyst can simultaneously explore data records and select items to
a label. By putting the analyst in the loop, higher accuracy can be
achieved in the resulting classifier. While initial results of
interactive visual labelling techniques are promising in the sense
that user labelling can improve supervised learning, many aspects of
these techniques are still largely unexplored.

This paper presents a study conducted using the mVis tool to compare
three interactive visualisations (similarity map, SPLOM with
scatterplot, and parallel coordinates) with each other and with active
learning for the purpose of labelling a multivariate dataset. The
results show that all three interactive visual labelling techniques
surpass active learning algorithms in terms of classifier accuracy and
that users subjectively prefer the similarity map over SPLOM with
scatterplot and parallel coordinates for labelling. Users also
employed different labelling strategies depending on the visualisation
being used.
Original languageEnglish
Pages (from-to)524-535
Number of pages12
JournalFrontiers of Information Technology & Electronic Engineering
Volume21
DOIs
Publication statusPublished - Apr 2020

    Fingerprint

Keywords

  • Interactive Visual Labelling
  • Visual Analytics
  • Active Learning

Cite this