HDTR: A Real-Time High-Definition Teeth Restoration Network for Arbitrary Talking Face Generation Methods

We propose A Real-Time High-Definition Teeth Restoration Network (HDTR-Net) to address talking face videos with blurred mouth in this work, which aims to improve clarity of talking face mouth and lip regions in real-time inference that correspond to given arbitrary talking face videos.

[Paper]

Recommondation of our works

This repo is maintaining by authors, if you have any questions, please contact us at issue tracker.

The official repository with Pytorch Our method can restorate teeth region for arbitrary face generation on images and videos

Test Results

Requirements

• python（We use version 3.7)
• PyTorch（We use version 1.13.1)
• opencv2
• ffmpeg

We conduct the experiments with 4 32G V100 on CUDA 10.2. For more details, please refer to the requirements.txt. We recommend to install pytorch firstly, and then run:

pip install -r requirements.txt

Generating test results

• Download the pre-trained model checkpoint Create the default folder ./checkpoint and put the checkpoint in it or get the CHECKPOINT_PATH, Then run the following

bash

CUDA_VISIBLE_DEVICES=0 python inference.py

To inference on other videos, please specify the --input_video option and see more details in code.

Citation and Star

Please cite the following paper and star this project if you use this repository in your research. Thank you!

@misc{li2023hdtrnet,
      title={HDTR-Net: A Real-Time High-Definition Teeth Restoration Network for Arbitrary Talking Face Generation Methods}, 
      author={Yongyuan Li and Xiuyuan Qin and Chao Liang and Mingqiang Wei},
      year={2023},
      eprint={2309.07495},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
},