Deep learning for BIPV segmentation on facades

Abstract. Building-integrated photovoltaics (BIPV) on facades are a significant but underutilized source of solar energy in urban environments. Automating the recognition of BIPV on facades through vision based systems can help guide design recommendations and extend solar asset maps. Unlike rooftop photovoltaic (PV) systems, facade BIPV recognition is difficult due to limited visibility in overhead imagery, high visual variability, and the absence of structured datasets. This study proposes a method based on deep learning (DL) for automated segmentation of BIPV panels on building facades using street-level and web images. A new dataset comprising 400 annotated BIPV projects was created, including detailed pixel-level masks and project attributes. Two model architectures, Mask Region-based Convolutional Neural Network (Mask R-CNN) and SegFormer, as well as human baselines are evaluated. The SegFormer model outperforms Mask R-CNN in pixel-level metrics. A user study conducted with human annotators without domain-specific expertise provides comparative insight into human performance, revealing common challenges in recognizing facade BIPV. The results demonstrate that DL models, trained specifically for this task, can segment BIPV panels more accurately than mean of human annotators, with SegFormer achieving an IoU of 0.69 compared to 0.42. The user study suggests that BIPV with satin finishes, invisible cells, and a PV-to-facade ratio of more than half challenge human recognition and therefore can be prioritized in visually sensitive areas. The outputs of the segmentation model are also utilized to estimate the BIPV energy yield. The annotated dataset and models are made available to facilitate future research.

Keywords: Building-integrated photovoltaics, deep learning, segmentation, computer vision, facade