Cross Field-Based Segmentation and Learning-Based Vectorization for Rectangular Windows

Xiangyu Zhuo*, Jiaojiao Tian*, Friedrich Fraundorfer*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Detection and vectorization of windows from building façades are important for building energy modeling, civil engineering, and architecture design. However, current applications still face the challenges of low accuracy and lack of automation. In this article we propose a new two-steps workflow for window segmentation and vectorization from façade images. First, we propose a cross field learning-based neural network architecture, which is augmented by a grid-based self-attention module for window segmentation from rectified façade images, resulting in pixel-wise window blobs. Second, we propose a regression neural network augmented by squeeze-and-excitation (SE) attention blocks for window vectorization. The network takes the segmentation results together with the original façade image as input, and directly outputs the position of window corners, resulting in vectorized window objects with improved accuracy. In order to validate the effectiveness of our method, experiments are carried out on four public façades image datasets, with results usually yielding a higher accuracy for the final window prediction in comparison to baseline methods on four datasets in terms of intersection over union score, F1 score, and pixel accuracy.
Original languageEnglish
Article number9935110
Pages (from-to)431-448
Number of pages18
JournalIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Publication statusPublished - 2 Nov 2023


  • Deep learning
  • façade parsing
  • vectorization
  • window segmentation

ASJC Scopus subject areas

  • Computers in Earth Sciences
  • Atmospheric Science


Dive into the research topics of 'Cross Field-Based Segmentation and Learning-Based Vectorization for Rectangular Windows'. Together they form a unique fingerprint.

Cite this