TY - JOUR
T1 - SSCNetViT
T2 - A Hybrid Siamese Sequential Classification-ViT Model for Kinship Recognition in Indonesia Facial Microexpression
AU - Fibriani, Ike
AU - Yuniarno, Eko Mulyanto
AU - Mardiyanto, Ronny
AU - Purnomo, Mauridhi Hery
N1 - Publisher Copyright:
© (2025), (Intelligent Network and Systems Society). All rights reserved.
PY - 2025
Y1 - 2025
N2 - Kinship recognition based on facial appearance is a challenging task with its unique complexity. Analyzing facial dynamics allows to assess the degree of similarity between individuals in the context of kinship. Although significant research has been done on basic microexpressions, there is currently a surge of interest in recognizing combined facial expressions of emotions in the field of image processing. The main focus of this study is to recognize kinship through a microexpression approach by proposing a hybrid model of Siamese Sequential Classification Network (SSCN) and vision transformer as a feature extractor instead of traditional convolution. This model combines the advantages of SSCN and Vision Transformer (ViT) models, the authors call it SSCNetViT, not only can capture global context information and present stronger learning ability, but also introduce SSCN inductive bias to improve generalization performance. The model is tested on independently collected datasets from the local Indonesian dataset (LaIndo) and the Families in the Wild (FIW) dataset. The results show that the L32 backbone achieves the highest average accuracy of 90.07%, with the peak performance in the BB class (99.5%) and the lowest in the FD class (84.0%). In comparison, the B16 and B32 backbones yield lower average accuracies of 88.0% and 83.3%, respectively, highlighting the effectiveness of the approach for kinship verification. Thus, our proposed SSCNetViT model with B16 quadratic feature fusion and multiplicative fusion strategies achieves the best performance and achieves better accuracy that outperforms previous state-of-the-art (SOTA) studies.
AB - Kinship recognition based on facial appearance is a challenging task with its unique complexity. Analyzing facial dynamics allows to assess the degree of similarity between individuals in the context of kinship. Although significant research has been done on basic microexpressions, there is currently a surge of interest in recognizing combined facial expressions of emotions in the field of image processing. The main focus of this study is to recognize kinship through a microexpression approach by proposing a hybrid model of Siamese Sequential Classification Network (SSCN) and vision transformer as a feature extractor instead of traditional convolution. This model combines the advantages of SSCN and Vision Transformer (ViT) models, the authors call it SSCNetViT, not only can capture global context information and present stronger learning ability, but also introduce SSCN inductive bias to improve generalization performance. The model is tested on independently collected datasets from the local Indonesian dataset (LaIndo) and the Families in the Wild (FIW) dataset. The results show that the L32 backbone achieves the highest average accuracy of 90.07%, with the peak performance in the BB class (99.5%) and the lowest in the FD class (84.0%). In comparison, the B16 and B32 backbones yield lower average accuracies of 88.0% and 83.3%, respectively, highlighting the effectiveness of the approach for kinship verification. Thus, our proposed SSCNetViT model with B16 quadratic feature fusion and multiplicative fusion strategies achieves the best performance and achieves better accuracy that outperforms previous state-of-the-art (SOTA) studies.
KW - Feature fusion
KW - Kinship recognition
KW - Micro-expressions
KW - Siamese sequential classification network
KW - Vision transformers
UR - http://www.scopus.com/inward/record.url?scp=85214273204&partnerID=8YFLogxK
U2 - 10.22266/ijies2025.0229.59
DO - 10.22266/ijies2025.0229.59
M3 - Article
AN - SCOPUS:85214273204
SN - 2185-310X
VL - 18
SP - 832
EP - 846
JO - International Journal of Intelligent Engineering and Systems
JF - International Journal of Intelligent Engineering and Systems
IS - 1
ER -