- ChildDiffusion: ChildDiffusion: Unlocking the Potential of Generative AI and Controllable Augmentations for Child Facial Data using Stable Diffusion and Large Language Models

ChildDiffusion: Discovering the Possibilities of Intelligent Transformations on Child Facial Data using Stable Diffusion and Large Language Models

Muhammad Ali Farooq1, Wang Yao1, Peter Corcoran1.

1C3I Group, School of Engineering, University of Galway
Model ChildDiffusion Sample Dataset ChildDiffusion Full Dataset Code arXiv

Example GIF

Given a text-prompt into ChildDiffusion model...

Introduction

This work contributes to the field of image synthesis by introducing a novel diffusion-based framework, ChildDiffusion, designed to render high-quality child facial data with advanced transformations and augmentations. This framework enables the creation of new synthetic datasets with the implementation of advanced data augmentation strategies, incorporating control-guided annotations and extensive text prompts generated by off-the-shelf large language models (LLMs), which significantly enriches the diversity and fidelity of synthetic child data. These methodologies enhance the utility of the data across various applications. Additionally, we have open-sourced a unique dataset of synthetic child race images generated through the ChildDiffusion framework, enhancing accessibility and furthering research in this domain.

The work is submitted in IEEE Transactions on Image Processing Journal.

MY ALT TEXT

Block Diagram

ControlNet is incorporated in conjuction with ChildDiffusion framework to embed complex transformations on child facial data as demonstrated in below figure.

MY ALT TEXT

ControlNet integration with ChildDiffusion

Intelligent transformation results on four different subjects by using ControlNet conditioning and complex textual guidance prompts extracted via GPT 3.5 LLM

Case 1 prompt: "Generate a series of images depicting the child's face transitioning through a range of emotions such as happiness, sadness, anger, surprise, and fear. Ensure the transformations are nuanced and realistic, capturing subtle changes in muscle movement, eye gaze, and mouth shape."

Case 2 prompt: "Develop a transformation to investigate the impact of environmental factors on facial morphology. Generate variations in the child’s facial features, such as skin tone, facial structure, and facial expressions, to simulate exposure to different climates, diets, and lifestyles."

Case 3 prompt: "Create an advanced transformation to explore the impact of hairstyle modifications on facial perception and identity. Generate facial images and apply sophisticated hairstyle manipulation techniques to experiment with a variety of haircuts, colours, and styling options."

Case 4 prompt: "Simulate the effects of facial expression dynamics on cheek contour and prominence. Generate facial images displaying a range of facial expressions, such as smiling, frowning, and pouting".

MY ALT TEXT

High Qulaity Synthetic Child Race Data Samples generated via ChildDiffusion Framework

MY ALT TEXT

African Race Data

MY ALT TEXT

Asian Race Data

MY ALT TEXT

Hispanic Race Data

MY ALT TEXT

White Race Data

MY ALT TEXT

South Asian/ Indian Race Data

Synthetic Face Rendering Results

MY ALT TEXT

Results genearted by employing various diffusion image sampling methods

MY ALT TEXT

Smart Transformation Results

Validation Results

MY ALT TEXT

Identity Validation

Video Presentation

Presentation Videos

Paper PDF

Related Links

Link of dataset samples.

Link of our models.

BibTeX

@ARTICLE{11021410,
  author={Farooq, Muhammad Ali and Yao, Wang and Corcoran, Peter},
  journal={IEEE Access}, 
  title={ChildDiffusion: Unlocking the Potential of Generative AI and Controllable Augmentations for Child Facial Data Using Stable Diffusion and Large Language Models}, 
  year={2025},
  volume={13},
  number={},
  pages={96616-96634},
  keywords={Diffusion models;Faces;Data models;Text to image;Image synthesis;Computational modeling;Buildings;Adaptation models;Training;Pipelines;T21;stable diffusion;synthetic data;GAN’s;generative AI;diffusion models},
  doi={10.1109/ACCESS.2025.3575964}}