Image/Video Analytics Customer Experience Sales and Marketing Automation

High Quality and Low Latency Face Super Resolution (FSR)

Quick and precise FSR for Facial Expression Recognition

Image showcasing the RESR-GAN model being deployed to ONNX.

The Client Technology company

Image of a AI bot performing image resolution enhancement.

A technology company that is a leading player in conversational AI and automation wanted a robust and efficient face super resolution (FSR) system that could transform smaller, low-quality facial images captured through web and CCTV cameras to usable sized, high-resolution images that can be utilized for recognizing facial expressions. The intent was to augment conversational AI with facial expression recognition to comprehend participant’s emotions during online meetings.

Top Benefits

Automatic transformation of low-quality facial images into high-resolution images
High pixel density and enhanced Image details that facilitate recognition of facial expressions and emotions

Book a Trial

Executive Summary

Industry Overview Disruption

Automation is the process of converting recurring, routine processes into autonomous operations that can be performed with zero or minimal human intervention. Automation improves efficiency, minimises errors, saves cost and labour, and can be applied to any sector. The market for automation software which was valued at USD 19.9 billion in 2021, is anticipated to reach USD 76.4 billion by 2030, increasing at a CAGR of 16.5% between 2022 and 2030. Artificial Intelligence is a key enabler of automation. Conversational AI, through technologies such as chatbots and virtual assistants, enables automatic comprehension, processing and response to customers. The global conversational AI market size is expected to almost triple from USD 6.8 billion in 2021 to USD 18.4 billion in 2026, growing at a CAGR of 21.8% during the forecast period.

Business Challenge Customer Experience

Intent and emotion inference is a key aspect of conversational AI and automation, and requires simultaneous processing of both audio and visual inputs for better precision. Understanding emotions from video inputs such as facial images requires high resolution images with enough detailing to recognize and comprehend facial expressions. Typical facial images from webcams that we obtain during online meetings are smaller in size and have a low resolution. For facial recognition AI models to work effectively, these images need to be transformed into usable sized high resolution images. Thus, it was necessary to develop a high quality and low latency face super resolution system that could be used with facial expression recognition models to decipher participant’s emotions during online meetings.

The Akaike Edge

Inbuilt libraries, DL models with transfer learning capabilities

Impact Delivered

Face Super Resolution achieved with 85% + accuracy
Fast face image transformation at a low latency of 13.89 ms per image
More than 1 million face images transformed

Book a Trial

Solution

Blend of Vision AI and Deep Learning

Leverage the power of Generative adversarial networks

Generative adversarial networks (GANs) are algorithmic architectures that use two neural networks, pitting one against the other (thus the “adversarial”) to discover and learn the regularities and patterns in the input data and generate new instances of data that are very close to the original. Thus, GAN is a good choice for face super resolution. After detailed analysis and experimentation, GFP-GAN (Generative Facial Prior GAN) and RESR-GAN (Real - Enhanced Super Resolution GAN), two popular models for face restoration and image and video super resolution respectively, were selected to build our FSR system.

Train the GFP_GAN and RESR-GAN models

The GFP-GAN and RESR-GAN models were trained with approximately 80K images across three different categories of emotions - Positive, Neutral, and Negative, to avoid any kind of biases. Based on experimental analysis, the RESR-GAN model, which offered an accuracy of more than 85% while preserving the identity, artefacts and facial expressions of the original image after upsampling, was chosen for productization. The RESR-GAN model also showed a low latency of 13.89 ms per image.

Training of the GFP-GAN and RESR-GAN models.

Deploy the deep residual learning model

The RESR-GAN model was deployed to ONNX, The Open Neural Network Exchange ecosystem. ONNX not only supports interoperability across frameworks, but also facilitates accelerated inferencing via runtimes and libraries designed to maximise performance across various hardware such as NVIDIA GPUs and Habana Gaudi.