LMFNet: Lightweight Multimodal Fusion Network for high-resolution remote sensing image segmentation

影响因子：7.5

发表刊物：Pattern Recognition

摘要：Despite the rapid evolution of semantic segmentation for land cover classification in high-resolution remote sensing imagery, integrating multiple data modalities such as Digital Surface Model (DSM), RGB, and Near-infrared (NIR) remains a challenge. Current methods often process only two types of data, missing out on the rich information that additional modalities can provide. Addressing this gap, we propose a novel Lightweight Multimodal data Fusion Network (LMFNet) to accomplish the tasks of fusion and semantic segmentation of multimodal remote sensing images. LMFNet uniquely accommodates various data types simultaneously, including RGB, NirRG, and DSM, through a weight-sharing, multi-branch vision transformer that minimizes parameter count while ensuring robust feature extraction. Our proposed multimodal fusion module integrates a Multimodal Feature Fusion Reconstruction Layer and Multimodal Feature Self-Attention Fusion Layer, which can reconstruct and fuse multimodal features. Our method achieves a mean Intersection over Union (mIoU) of 85.09% on the US3D dataset, marking a significant improvement over existing methods. We also studied the scalability of our method, directly extending the input modality to the SAR and hyperspectral fields. Our experimental results on the C2Seg dataset show that our method has generalization applicability to data of various modalities.

合写作者：Chenxi Liu,Jiaqi Wang,Xiaoliang Tan,Wenlin Zhou,Chanjuan He

论文类型：期刊论文

通讯作者：Xiaodong Zhang

文献类型：J

卷号：164

是否译文：否

发表时间：2025-03-15

收录刊物：SCI

发布期刊链接：https://www.sciencedirect.com/science/article/abs/pii/S0031320325002390