A sketch-aided retrieval approach for incomplete 3D objects

S. Lengauer*, A. Komar, A. Labrada, S. Karl, E. Trinkl, R. Preiner, B. Bustos, T. Schreck

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

With the growing amount of digital collections of visual CH data being available across different repositories, it becomes increasingly important to provide archaeologists with means to find relations and cross-correspondences between different digital records. In principle, existing shape- and image-based similarity search methods can aid such domain analysis tasks. However, in practice, visual object data are given in different modalities, and often only in incomplete or fragmented state, posing a particular challenge for conventional similarity search approaches. In this paper we introduce a methodology and system for cross-modal visual search in CH object data that addresses these challenges. Specifically, we propose a new query modality based on 3D views enhanced by user sketches (3D+sketch). This allows for adding new context to the search, which is useful e.g., for searching based on incomplete query objects, or for testing hypotheses on existence of certain shapes in a collection. We present an appropriately designed workflow for constructing query views from incomplete 3D objects enhanced by a user sketch, based on shape completion and texture inpainting. Visual cues additionally help users compare retrieved objects with the query. The proposed approach extends on a previously presented retrieval system by introducing improved retrieval methods, an extended evaluation including retrieval in a larger and richer data collection, and enhanced interactive search weight specification. We demonstrate the feasibility and potential of our approach to support analysis of domain experts in Archaeology and the field of CH in general.

Original languageEnglish
Pages (from-to)111-122
Number of pages12
JournalComputers & Graphics
Volume87
DOIs
Publication statusPublished - 2020

Keywords

  • Image processing
  • Information storage and retrieval
  • Information systems
  • Shapes

ASJC Scopus subject areas

  • Engineering(all)
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Fields of Expertise

  • Information, Communication & Computing

Fingerprint

Dive into the research topics of 'A sketch-aided retrieval approach for incomplete 3D objects'. Together they form a unique fingerprint.

Cite this