TY - JOUR
T1 - AdaCrossNet
T2 - Adaptive Dynamic Loss Weighting for Cross-Modal Contrastive Point Cloud Learning
AU - Putra, Oddy Virgantara
AU - Ogata, Kohichi
AU - Yuniarno, Eko Mulyanto
AU - Purnomo, Mauridhi Hery
N1 - Publisher Copyright:
© (2025), (Intelligent Network and Systems Society). All rights reserved.
PY - 2025
Y1 - 2025
N2 - Manual annotation of large-scale point cloud datasets is laborious due to their irregular structure. While cross-modal contrastive learning methods such as CrossPoint and CrossNet have progressed in utilizing multimodal data for self-supervised learning, they still suffer from instability during training caused by the static weighting of intra-modal (IM) and cross-modal (CM) losses. These static weights fail to account for the varying convergence rates of different modalities. We propose AdaCrossNet, a novel self-supervised learning framework for point cloud understanding that utilizes a dynamic weight adjustment mechanism for IM and CM contrastive learning. AdaCrossNet learns representations by simultaneously enhancing the alignment between 3-D point clouds and their associated 2D-rendered images within a common latent space. Our dynamic weight adjustment mechanism adaptively balances the contributions of IM and CM losses during training, guided by the convergence behavior of each modality. To ensure stability in the training process, we employ an exponentially weighted moving average (EWMA) to smooth the weight updates. We experimented with benchmark datasets, ModelNet40, ShapeNetPart, and ScanObjectNN. The results demonstrate that AdaCrossNet achieves superiority over other methods, with 91.4% accuracy on the ModelNet40 classification task. While on the segmentation task, AdaCrossNet achieved the mIoU score of 85.1% on the ShapeNetPart segmentation task. Additionally, AdaCrossNet, when combined with the DGCNN backbone, showed significant improvements in the ScanObjectNN dataset with 82.1% accuracy. Our method boosts training efficiency while increasing the generalizability of the learned representations across downstream tasks.
AB - Manual annotation of large-scale point cloud datasets is laborious due to their irregular structure. While cross-modal contrastive learning methods such as CrossPoint and CrossNet have progressed in utilizing multimodal data for self-supervised learning, they still suffer from instability during training caused by the static weighting of intra-modal (IM) and cross-modal (CM) losses. These static weights fail to account for the varying convergence rates of different modalities. We propose AdaCrossNet, a novel self-supervised learning framework for point cloud understanding that utilizes a dynamic weight adjustment mechanism for IM and CM contrastive learning. AdaCrossNet learns representations by simultaneously enhancing the alignment between 3-D point clouds and their associated 2D-rendered images within a common latent space. Our dynamic weight adjustment mechanism adaptively balances the contributions of IM and CM losses during training, guided by the convergence behavior of each modality. To ensure stability in the training process, we employ an exponentially weighted moving average (EWMA) to smooth the weight updates. We experimented with benchmark datasets, ModelNet40, ShapeNetPart, and ScanObjectNN. The results demonstrate that AdaCrossNet achieves superiority over other methods, with 91.4% accuracy on the ModelNet40 classification task. While on the segmentation task, AdaCrossNet achieved the mIoU score of 85.1% on the ShapeNetPart segmentation task. Additionally, AdaCrossNet, when combined with the DGCNN backbone, showed significant improvements in the ScanObjectNN dataset with 82.1% accuracy. Our method boosts training efficiency while increasing the generalizability of the learned representations across downstream tasks.
KW - Adaptive weighting
KW - Contrastive learning
KW - Deep learning
KW - Point cloud understanding
KW - Self-Supervised learning
UR - http://www.scopus.com/inward/record.url?scp=85214251665&partnerID=8YFLogxK
U2 - 10.22266/ijies2025.0229.11
DO - 10.22266/ijies2025.0229.11
M3 - Article
AN - SCOPUS:85214251665
SN - 2185-310X
VL - 18
SP - 134
EP - 146
JO - International Journal of Intelligent Engineering and Systems
JF - International Journal of Intelligent Engineering and Systems
IS - 1
ER -