Effective Use of BERT in Graph Embeddings for Sparse Knowledge Graph Completion

Xinglan Liu, Hussain Hussain, Houssam Razouk, Roman Kern

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

Graph embedding methods have emerged as effective solutions for knowledge graph completion. However, such methods are typically tested on benchmark datasets such as Freebase, but show limited performance when applied on sparse knowledge graphs with orders of magnitude lower density. To compensate for the lack of structure in a sparse graph, low dimensional representations of textual information such as word2vec or BERT embeddings have been used. This paper proposes a BERT-based method (BERT-ConvE), to exploit transfer learning of BERT in combination with a convolutional network model ConvE. Comparing to existing text-aware approaches, we effectively make use of the context dependency of BERT embeddings through optimizing the features extraction strategies. Experiments on ConceptNet show that the proposed method outperforms strong baselines by 50% on knowledge graph completion tasks. The proposed method is suitable for sparse graphs as also demonstrated by empirical studies on ATOMIC and sparsified-FB15k-237 datasets. Its effectiveness and simplicity make it appealing for industrial applications.
Original languageEnglish
Title of host publicationProceedings of the 37th ACM/SIGAPP Symposium on Applied Computing, SAC 2022
Place of PublicationNew York, NY, USA
PublisherAssociation of Computing Machinery
Pages799–802
Number of pages4
ISBN (Electronic)9781450387132
DOIs
Publication statusPublished - 25 Apr 2022
Event37th ACM/SIGAPP Symposium On Applied Computing: SAC 2022 - Virtuell, United States
Duration: 25 Apr 202229 Apr 2022

Publication series

NameProceedings of the ACM Symposium on Applied Computing

Conference

Conference37th ACM/SIGAPP Symposium On Applied Computing
Abbreviated titleSAC 2022
Country/TerritoryUnited States
CityVirtuell
Period25/04/2229/04/22

Keywords

  • BERT
  • knowledge graph embedding
  • sparse knowledge graph
  • language model
  • context aware embedding

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Effective Use of BERT in Graph Embeddings for Sparse Knowledge Graph Completion'. Together they form a unique fingerprint.

Cite this