An Adaptive Annotation Approach for Biomedical Entity and Relation Recognition

Seid Muhie Yimam, Chris Biemann, Ljiljana Majnaric, Sefket Sabanovic, Andreas Holzinger

Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

Abstract

In this article, we demonstrate the impact of interactive machine learning: we develop biomedical entity recognition dataset using a human-into-the-loop approach. In contrary to classical machine learning, human-in-the-loop approaches do not operate on predefined training or test sets, but assume that human input regarding system improvement is supplied iteratively. Here, during annotation, a machine learning model is built on previous annotations and used to propose labels for subsequent annotation. To demonstrate that such interactive and iterative annotation speeds up the development of quality dataset annotation, we conduct three experiments. In the first experiment, we carry out an iterative annotation experimental simulation and show that only a handful of medical abstracts need to be annotated to produce suggestions that increase annotation speed. In the second experiment, clinical doctors have conducted a case study in annotating medical terms documents relevant for their research. The third experiment explores the annotation of semantic relations with relation instance learning across documents. The experiments validate our method qualitatively and quantitatively, and give rise to a more personalized, responsive information extraction technology.
Originalspracheenglisch
Seiten (von - bis)157-168
FachzeitschriftBrain Informatics
Jahrgang3
Ausgabenummer3
DOIs
PublikationsstatusVeröffentlicht - 31 Okt 2016

Fingerprint

Learning systems
Experiments
Labels
Semantics

Schlagwörter

    ASJC Scopus subject areas

    • Artificial intelligence

    Fields of Expertise

    • Information, Communication & Computing

    Treatment code (Nähere Zuordnung)

    • Basic - Fundamental (Grundlagenforschung)

    Dies zitieren

    An Adaptive Annotation Approach for Biomedical Entity and Relation Recognition. / Yimam, Seid Muhie; Biemann, Chris; Majnaric, Ljiljana; Sabanovic, Sefket; Holzinger, Andreas.

    in: Brain Informatics, Jahrgang 3, Nr. 3, 31.10.2016, S. 157-168.

    Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

    Yimam, Seid Muhie ; Biemann, Chris ; Majnaric, Ljiljana ; Sabanovic, Sefket ; Holzinger, Andreas. / An Adaptive Annotation Approach for Biomedical Entity and Relation Recognition. in: Brain Informatics. 2016 ; Jahrgang 3, Nr. 3. S. 157-168.
    @article{35996198df68427c97deec765551aec1,
    title = "An Adaptive Annotation Approach for Biomedical Entity and Relation Recognition",
    abstract = "In this article, we demonstrate the impact of interactive machine learning: we develop biomedical entity recognition dataset using a human-into-the-loop approach. In contrary to classical machine learning, human-in-the-loop approaches do not operate on predefined training or test sets, but assume that human input regarding system improvement is supplied iteratively. Here, during annotation, a machine learning model is built on previous annotations and used to propose labels for subsequent annotation. To demonstrate that such interactive and iterative annotation speeds up the development of quality dataset annotation, we conduct three experiments. In the first experiment, we carry out an iterative annotation experimental simulation and show that only a handful of medical abstracts need to be annotated to produce suggestions that increase annotation speed. In the second experiment, clinical doctors have conducted a case study in annotating medical terms documents relevant for their research. The third experiment explores the annotation of semantic relations with relation instance learning across documents. The experiments validate our method qualitatively and quantitatively, and give rise to a more personalized, responsive information extraction technology.",
    keywords = "Interactive Annotation, Human-in-the-loop, Doctor-in-the-loop, Biomedical Entity Recognition, Knowledge Discovery, data mining, Relation learning",
    author = "Yimam, {Seid Muhie} and Chris Biemann and Ljiljana Majnaric and Sefket Sabanovic and Andreas Holzinger",
    year = "2016",
    month = "10",
    day = "31",
    doi = "10.1007/s40708-016-0036-4",
    language = "English",
    volume = "3",
    pages = "157--168",
    journal = "Brain Informatics",
    issn = "2198-4018",
    publisher = "SpringerOpen",
    number = "3",

    }

    TY - JOUR

    T1 - An Adaptive Annotation Approach for Biomedical Entity and Relation Recognition

    AU - Yimam, Seid Muhie

    AU - Biemann, Chris

    AU - Majnaric, Ljiljana

    AU - Sabanovic, Sefket

    AU - Holzinger, Andreas

    PY - 2016/10/31

    Y1 - 2016/10/31

    N2 - In this article, we demonstrate the impact of interactive machine learning: we develop biomedical entity recognition dataset using a human-into-the-loop approach. In contrary to classical machine learning, human-in-the-loop approaches do not operate on predefined training or test sets, but assume that human input regarding system improvement is supplied iteratively. Here, during annotation, a machine learning model is built on previous annotations and used to propose labels for subsequent annotation. To demonstrate that such interactive and iterative annotation speeds up the development of quality dataset annotation, we conduct three experiments. In the first experiment, we carry out an iterative annotation experimental simulation and show that only a handful of medical abstracts need to be annotated to produce suggestions that increase annotation speed. In the second experiment, clinical doctors have conducted a case study in annotating medical terms documents relevant for their research. The third experiment explores the annotation of semantic relations with relation instance learning across documents. The experiments validate our method qualitatively and quantitatively, and give rise to a more personalized, responsive information extraction technology.

    AB - In this article, we demonstrate the impact of interactive machine learning: we develop biomedical entity recognition dataset using a human-into-the-loop approach. In contrary to classical machine learning, human-in-the-loop approaches do not operate on predefined training or test sets, but assume that human input regarding system improvement is supplied iteratively. Here, during annotation, a machine learning model is built on previous annotations and used to propose labels for subsequent annotation. To demonstrate that such interactive and iterative annotation speeds up the development of quality dataset annotation, we conduct three experiments. In the first experiment, we carry out an iterative annotation experimental simulation and show that only a handful of medical abstracts need to be annotated to produce suggestions that increase annotation speed. In the second experiment, clinical doctors have conducted a case study in annotating medical terms documents relevant for their research. The third experiment explores the annotation of semantic relations with relation instance learning across documents. The experiments validate our method qualitatively and quantitatively, and give rise to a more personalized, responsive information extraction technology.

    KW - Interactive Annotation

    KW - Human-in-the-loop

    KW - Doctor-in-the-loop

    KW - Biomedical Entity Recognition

    KW - Knowledge Discovery

    KW - data mining

    KW - Relation learning

    UR - http://link.springer.com/journal/40708

    U2 - 10.1007/s40708-016-0036-4

    DO - 10.1007/s40708-016-0036-4

    M3 - Article

    VL - 3

    SP - 157

    EP - 168

    JO - Brain Informatics

    JF - Brain Informatics

    SN - 2198-4018

    IS - 3

    ER -