Wang, Hengfei (2025). Gaze-controllable face generation. University of Birmingham. Ph.D.
![[img]](http://etheses.bham.ac.uk//style/images/fileicons/text.png) |
Wang2025PhD.pdf
Text - Accepted Version Available under License All rights reserved. Download (20MB) |
Abstract
Gaze-controllable face generation systems produce facial images with adjustable gazes, enabling a wide range of applications such as virtual reality, digital humans, and computer-generated filmmaking. Traditional methods achieve gaze control through image replacement or graphical modeling techniques. However, to make gaze-controllable face generation more accessible and practical, researchers are increasingly focusing on leveraging deep neural networks, particularly generative models, in both 2D and 3D frameworks. My work addresses three outstanding problems regarding quality, speed, and usability in this field: 1) It is hard to model both rigid eyeball rotation and non-rigid eyelid deformation in a unified 3D representation; 2) The inference of 3D gaze-controllable generation models is still far from being real-time; 3) Using explicit gaze labels to control gaze is inconvenient for general use.
In this dissertation, I propose three approaches to tackle these problems separately: modeling a hybrid transformation from observation space to canonical space with one unified 3D model for high-fidelity generation, distilling face shape prior into an efficient 3D representation for real-time generation, controlling gaze image generation with the guidance of natural language for convenient use.
I present three key contributions in this dissertation. First, I introduce an eye-controllable face NeRF model handling rigid eyeball rotation and non-rigid eyelid deformation. The model can generate high-fidelity images under novel head poses and gaze directions. Second, I propose a real-time 3D-aware gaze redirection module generating multi-view gaze images from a single face image. It takes 61ms to process the input and generate results, outperforming other methods by a large margin. Third, I present the first text-to-gaze dataset and a novel framework enabling language-guided gaze-controllable face generation. It provides a more natural way to control gaze image generation.
| Type of Work: | Thesis (Doctorates > Ph.D.) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Award Type: | Doctorates > Ph.D. | |||||||||
| Supervisor(s): |
|
|||||||||
| Licence: | All rights reserved | |||||||||
| College/Faculty: | Colleges > College of Engineering & Physical Sciences | |||||||||
| School or Department: | School of Computer Science | |||||||||
| Funders: | Other | |||||||||
| Other Funders: | China Scholarship Council | |||||||||
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science | |||||||||
| URI: | http://etheses.bham.ac.uk/id/eprint/16110 |
Actions
 |
Request a Correction |
 |
View Item |
Downloads
Downloads per month over past year