Small facial image dataset augmentation using conditional GANs based on incomplete edge feature input

View article
PeerJ Computer Science

Main article text

 

Introduction

  • In order to deal with the problem of distortions in the images generated by conditional GANs due to using very small training data and sparse conditional input for diverse data augmentation, a new conditional GAN framework has been proposed, which converts the original incomplete edges into new conditional inputs in an interim domain for refining images and thus alleviates distortions caused by small training data and incomplete conditional edges.

  • For the first time, the proposed method uses the mixture of pixel values of both binary images and segmentation masks to enhance the conditional input in an interim domain for refining images, which can integrate facial components, including eyes, nose, mouth, etc., so as to introduce diversity and enhance the quality of the images generated by conditional GANs trained using a very small training dataset.

  • A facial image augmentation method using conditional GANs has been proposed, which can generate photorealistic facial images of diversity from incomplete edges or hand-drawn sketches. Compared with existing edge-to-image translation methods without ideal conditional inputs, the proposed method is tolerant to various incomplete edges as conditional inputs and able to generate diverse images of higher quality in terms of Fréchet Inception Distance (FID) (Heusel et al., 2017) and Kernel Inception Distance (KID) (Binkowski et al., 2018).

Methods

The proposed conditional GAN framework

Image pre-processing and refining

Edge extraction

Adoption of an interim domain

Model training and loss functions

Conditional adversarial loss

Feature matching loss

Overall loss

Experiments with the proposed conditional GAN framework

Data preparation

Implementation details

Results and performance evaluation

Diversity in facial image augmentation using the proposed conditional GAN

Qualitative comparison

Quantitative comparison

Conclusions and Future Work

Supplemental Information

Augmented facial images with hand drawing lines

DOI: 10.7717/peerj-cs.760/supp-2

Augmented facial images with blending features

DOI: 10.7717/peerj-cs.760/supp-3

Augmented facial images with the Canny edge parameters of 0.1, 0.2 and 0.3

DOI: 10.7717/peerj-cs.760/supp-4

Augmented facial images with the Canny edge parameters of 0.4, 0.5 and 0.6

DOI: 10.7717/peerj-cs.760/supp-5

Additional Information and Declarations

Competing Interests

The authors declare there are no competing interests.

Author Contributions

Shih-Kai Hung conceived and designed the experiments, performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, and approved the final draft.

John Q. Gan conceived and designed the experiments, performed the experiments, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Data Availability

The following information was supplied regarding data availability:

The CelebA dataset is available at: https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html.

The CelebAMask-HQ dataset is available at GitHub: https://github.com/switchablenorms/CelebAMask-HQ.

The raw measurements and source code are available in the Supplementary Files.

Funding

The authors received no funding for this work.

3 Citations 1,641 Views 138 Downloads