VToonify: Transforming High-Resolution Portrait Video with Tunable Cartoon Style Transfer

Opportunity

In our ever-evolving digital landscape, there is a growing appetite for creative and personalised content. Over the last few years, video content has rapidly entrenched itself as a mainstay in our social media feeds, captivating audiences and fuelling the demand for innovative video editing techniques, such as portrait video style transfer. Our technology, VToonify, empowers users with a seamless means to infuse their videos with artistic flair. It opens up a world of possibilities, enabling users to effortlessly apply trending visual styles to their portraits, thereby transforming ordinary faces into mesmerising, cartoon-like characters. Whether for personal expression, entertainment, social media, marketing, or branding, this technology offers endless possibilities for customisation and creativity.

Technology

Our VToonify technology leverages state-of-the-art neural networks to transform facial videos into stunning cartoon artworks. Specifically, VToonify leverages the mid- and high-resolution layers of StyleGAN, a style-based generator architecture for generative adversarial networks, to render high-quality artistic portraits based on the multi-scale content features extracted by an encoder to better preserve the frame details. With fine-tuned control over parameters, users can customise the degree of toonification, from subtle enhancements to whimsical exaggerations to map the input image from the real face domain to the artistic portrait domain.

Figure 1: Tunable Cartoon Style Transfer.

Applications & Advantages

1. Entertainment and Gaming: Enhance character creation by adding unique styles or toon effects, making games and animations more visually appealing.
2. Social Media: Stand out with eye-catching profile pictures and posts, reflecting individuality or trending visual styles.
3. Marketing and Advertising: Create attention-grabbing campaigns with stylised or toonified brand mascots or spokespersons.
4. Digital Artistry: Fuel creativity by offering a unique tool for artists to experiment and create distinctive digital portraits.
5. Education and Training: Develop engaging e-learning content with visually compelling avatars and characters.

References

1. https://www.mmlab-ntu.com/project/vtoonify/
2. Shuai Yang, Liming Jiang, Ziwei Liu and Chen Change Loy, VToonify: Controllable High-Resolution Portrait Video Style Transfer, ACM Transactions on Graphics (TOG), Volume 41, Issue 6, article no.: 203, pages 1-15, 2022, ACM New York, NY, USA, DOI: 10.1145/3550454.3555437, https://arxiv.org/pdf/2209.11224.