AdaCrossNet: Adaptive Dynamic Loss Weighting for Cross-Modal Contrastive Point Cloud Learning

Oddy Virgantara Putra, Kohichi Ogata, Eko Mulyanto Yuniarno, Mauridhi Hery Purnomo*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Manual annotation of large-scale point cloud datasets is laborious due to their irregular structure. While cross-modal contrastive learning methods such as CrossPoint and CrossNet have progressed in utilizing multimodal data for self-supervised learning, they still suffer from instability during training caused by the static weighting of intra-modal (IM) and cross-modal (CM) losses. These static weights fail to account for the varying convergence rates of different modalities. We propose AdaCrossNet, a novel self-supervised learning framework for point cloud understanding that utilizes a dynamic weight adjustment mechanism for IM and CM contrastive learning. AdaCrossNet learns representations by simultaneously enhancing the alignment between 3-D point clouds and their associated 2D-rendered images within a common latent space. Our dynamic weight adjustment mechanism adaptively balances the contributions of IM and CM losses during training, guided by the convergence behavior of each modality. To ensure stability in the training process, we employ an exponentially weighted moving average (EWMA) to smooth the weight updates. We experimented with benchmark datasets, ModelNet40, ShapeNetPart, and ScanObjectNN. The results demonstrate that AdaCrossNet achieves superiority over other methods, with 91.4% accuracy on the ModelNet40 classification task. While on the segmentation task, AdaCrossNet achieved the mIoU score of 85.1% on the ShapeNetPart segmentation task. Additionally, AdaCrossNet, when combined with the DGCNN backbone, showed significant improvements in the ScanObjectNN dataset with 82.1% accuracy. Our method boosts training efficiency while increasing the generalizability of the learned representations across downstream tasks.

Original languageEnglish
Pages (from-to)134-146
Number of pages13
JournalInternational Journal of Intelligent Engineering and Systems
Volume18
Issue number1
DOIs
Publication statusPublished - 2025

Keywords

  • Adaptive weighting
  • Contrastive learning
  • Deep learning
  • Point cloud understanding
  • Self-Supervised learning

Fingerprint

Dive into the research topics of 'AdaCrossNet: Adaptive Dynamic Loss Weighting for Cross-Modal Contrastive Point Cloud Learning'. Together they form a unique fingerprint.

Cite this