INVITATION: A Framework for Enhancing UAV Image Semantic Segmentation Accuracy through Depth Information Fusion

影响因子：4.0

DOI码：10.1109/LGRS.2025.3534994

发表刊物：IEEE Geoscience and Remote Sensing Letters

摘要：With the increasing use of uncrewed aerial vehicles (UAVs), improving the accuracy of semantic segmentation is becoming critical. Depth information preserves geometric structure, serving as an invaluable supplement to color-rich UAV imagery. Inspired by this, we proposed a novel framework named INVITATION, which exclusively takes original UAV imagery as input, yet is capable of obtaining complemented depth information and fusing into RGB semantic segmentation models effectively, thereby enhancing UAV semantic segmentation accuracy. Concretely, this framework supports two distinct depth generation approaches: high-precision multiview stereo (MVS) depth reconstruction using multiple views or video sequences via structure from motion (SfM) and monocular depth estimation using individual images. Our empirical evaluations conducted on the UAVid dataset showed that mIoU metric of INVITATION used precise reconstructed depth maps via MVS improved from 66.02% to 70.57%, while used depth predictions from pretrained models reached 69.69%, which supports the effectiveness of extracting and fusing depth information from original imagery in enhancing UAV semantic segmentation. This study explores a novel approach to acquire UAV multimodal information at low data cost, highlights the advantages of incorporating depth information into UAV semantic analysis, and paves the way for further studies on the integration of multimodal UAV information. Our code is available at https://github.com/CVEO/INVITATION

合写作者：Wenlin Zhou,Jiaqi Wang,Qingyuan Yang,Xiaoliang Tan

论文类型：期刊论文

通讯作者：Guanzhou Chen

卷号：22

ISSN号：1545-598X

是否译文：否

发表时间：2025-01-30

收录刊物：SCI、EI

发布期刊链接：https://ieeexplore.ieee.org/abstract/document/10858079