Cool Machine Learning Models to Empower AI Projects

CatalyzeX is the largest collection of machine learning (ML) models and code to empower AI-based projects and provide paid opportunities for ML experts.

We’ve compiled some of the most interesting and thought-provoking ML models that you may find useful and applicable to your artificial intelligence (AI) based project.

Latest From Facebook AI Researchers: Codec Avatars

Research Team: Alexander Richard, Colin Lea, Shugao Ma, Juergen Gall, Fernando de la Torre, Yaser Sheikh 

Codec Avatars is a recent new class of photorealistic face models that closely match the geometry and texture of a person in 3D (i.e., for Virtual Reality) and are almost indistinguishable from the video.

It is the very first approach to animating these parametric models in real-time that can be deployed on conventional VR equipment using sound and/or eye-tracking. The project goal is to show expressive conversations between people that demonstrate important social cues, such as laughter and excitement, solely due to latent cues in our lossy input signals.

The researchers have collected over 5 hours of high frame rate 3D face scans from three participants, including traditional neutral speech as well as expressive and spoken languages. They’re investigating a multimodal fusion approach that dynamically determines which encoding sensor should animate which parts of the face at any time.

Automated Capture of Animal Pose

Research Team: Marc Badger, Yufu Wang, Adarsh Modh, Ammon Perkes, Nikos Kolotouros, Bernd G. Pfrommer, Marc F. Schmidt, Kostas Daniilidis 

Automatically capturing animal poses is changing the way we study neuroscience and social behavior. The movement carries important social cues, but current techniques cannot reliably assess the posture and shape of animals, especially social animals such as birds, which are often obscured by each other and by objects in the environment.

To address this issue, for the first time ever, researchers have presented a model and multiple view optimization approaches you can use to capture the unique shape and space of the pose as rendered by live birds.

Then, they have introduced a pipeline and experiments for regression of keypoints, masks, poses, and shapes that reconstruct the exact poses of birds from individual images. Finally, they’ve provided extensive annotations of keypoints and masks from a group of 15 social birds housed together in an outdoor aviary.

Supervised Color Aware GAN for Controllable Makeup Transfer

Research Team: Robin Kips, Pietro Gori, Matthieu Perrot, Isabelle Bloch 

While existing makeup style transfer models perform image synthesis whose results cannot be explicitly controlled, the ability to continuously change makeup color is a desirable property for virtual fitting applications. The researchers propose a new formulation of the problem of transferring the makeup style in order to learn how to synthesize the makeup style with controlled color.

They present CA-GAN, a generative model that learns to change the color of specific objects (like lips or eyes) in an image to an arbitrary target color while preserving the background. Since colored labels are rare and expensive, our method uses loosely supervised learning for conditional GANs.

This allows AI practitioners to study the controlled synthesis of complex objects and it only requires a weak proxy for the image attribute we want to change. Finally, they’ve presented for the first time a quantitative analysis of makeup style transfer and color management.

DeepFacePencil: Creating Face Images from Freehand Sketches

Research Team: Yuhang Li, Xuejin Chen, Binxin Yang, Zihan Chen, Zhihua Cheng, Zheng-Jun Zha 

The project aims to explore the problem of creating photorealistic face images from hand-drawn sketches. Existing image-to-image conversion techniques require a large-scale set of paired sketches and images to be observed. They usually use synthesized edge maps of face images as training data.

However, these synthesized edge maps strictly match the edges of the corresponding face images, which limits their generalizability to real hand-drawn sketches with a huge variety of strokes.

To solve this problem, researchers have created DeepFacePencil, an efficient tool that can create photorealistic face images from hand-drawn sketches based on a new dual-generator image transformation network during training.

An innovative Spatial Attention Poolong (SAP) is designed to adaptively handle the distortion of strokes that change in space to support different stroke styles and different levels of detail.

Having conducted extensive experiments, researchers came up with results that demonstrate the superiority of their model over existing methods, both in terms of image quality and generalization of the model over hand-drawn sketches.

To be continued…

Stay tuned with Software Focus!