Project's information

Project's title Study on Human Visual Tracking Core Technology based on Multi-modal Deep Neural Network
Project’s code QTKR01.01/20-21
Research hosting institution Institute of Information Technology
Coordinating unit, co-chair Chosun University, Korea
Project leader’s name Dr. Dung Nguyen Duc
Project duration 01/06/2020 - 30/06/2022
Project’s budget 200 million VND
Classify Fair
Goal and objectives of the project
Study on Human Visual Tracking Core Technology based on Multi-modal Deep Neural Network.
Main results

Research: 01 Publication
In Seop Na, Chung Tran, Dung Nguyen & Sang Dinh, Facial UV map completion for pose-invariant face recognition: a novel adversarial approach based on coupled attention residual UNets, Human-centric Computing and Information Sciences, volume 10, Article number: 45 (2020) (SCIE)
Education and training: 01 Student
Student: Phan Thi Thu Trang, ‘Hướng tới xây dựng hệ thống camera thông minh giám sát nhân viên và khách hàng’, Supervisor: TS. Dinh Viet Sang

Novelty and actuality and scientific meaningfulness of the results

We introduce a novel generative model called Attention ResCUNet-GAN to generate complete facial UV maps, which allows us to synthesize various faces of arbitrary poses and improve pose-invariant face recognition performance. We leverage the residual connections in ResNet, intra-block and extra-block feature fusion in coupled UNets to enhance the generator. The skip connections within each U-Net are amplifed with attention gates, while the contextual feature maps from two U-Nets are fused with trainable scalar weights. We jointly train global and local adversarial losses with identity preserving loss. The experiments show that the proposed Attention ResCUNet-GAN outperforms the original UV-GAN by order of magnitude in terms of both reconstruction metrics and the performance on the pose-invariant face verifcation task. We also developed a method based on the CenterNet model to detect and track people. The model is trained on an extensive dataset combining the benchmarks and self-collected images.

Products of the project

- Publications:
In Seop Na, Chung Tran, Dung Nguyen & Sang Dinh, Facial UV map completion for pose-invariant face recognition: a novel adversarial approach based on coupled attention residual UNets, Human-centric Computing and Information Sciences, volume 10, Article number: 45 (2020) (SCIE)

Recommendations

That would be much appreciated if the leaders of Vietnam Academy of Science and Technology make consideration, evaluation and acceptance for the research results.

Images of project
1695107885611-Screen Shot 2023-09-19 at 2.15.36 PM.png