基于联合表示学习的跨视觉模态生物识别研究

CROSS-VISUALMODALBIOLOGICALRECOGNITION BASED ON JOINT REPRESENTATION LEARNING

  • 摘要: 提出基于联合表示学习的跨视觉模态生物识别方法。具体而言,针对跨人脸-眼周识别问题,提出深度卷积自注意力网络(Deep Convolutional Self-attention Network,DCSAN)和跨视觉模态对比损失函数(Cross-visual-modality Contrast Loss,CVMC)。DCSAN采用深度卷积层来提取不同模态图像块中的纹理特征,实现局部联合表示学习;采用深度卷积多头自注意模块(Deep Convolutional Self-attention Module,DWC-MHSAM)来建模跨模态的全局依赖关系。CVMC损失有助于处理跨模态负样本对,并有助于区分同一模态下不同个体之间高相似度的人脸和眼周图像块。采用Ethnic、FaceScrub和IMDB数据集进行测试,结果表明以人脸作为查询集,最优识别率达到75.46%;以眼周作为查询集,最优识别率达到76.36%。

     

    Abstract: This paper proposes a cross- visual modal biological recognition method based on joint representation learning. Specifically, for the cross face- eye area recognition problem, we introduced a deep convolutional self- attention network (DCSAN) and a cross- visual- modality contrast loss (CVMC). DCSAN utilized deep convolutional layers to extract texture features from face and eye area image patches, enabling local joint representation learning. Additionally, it employed deep convolutional multi- head self- attention modules (DWC- MHSAM) to model global dependencies between face and eye area regions. The introduction of the CVMC loss helped handle cross- modal negative sample pairs, and distinguished highly similar face and eye area image patches belonging to different individuals within the same modality. We conducted experiments on the Ethnic, FaceScrub, and IMDB datasets. The results demonstrate that when using face as the query set, the optimal recognition rate reached 75.46%, while using eye area as the query set achieved an optimal recognition rate of 76.36%.

     

/

返回文章
返回