GHOST 2.0：生成式高保真一次性头部迁移

摘要

. 尽管面部交换任务最近在学术界引起了广泛关注，但与之相关的头部交换问题却仍鲜有研究。除了肤色迁移外，头部交换还带来了额外的挑战，例如在合成过程中需要保留整个头部的结构信息，以及填补交换后的头部与背景之间的空隙。本文中，我们通过GHOST 2.0系统解决了这些问题，该系统包含两个针对特定问题的模块。首先，我们引入了增强版的Aligner模型用于头部重现，该模型能在多个尺度上保留身份信息，并对极端姿态变化具有鲁棒性。其次，我们采用了一个Blender模块，通过肤色迁移和修复不匹配区域，将重现的头部无缝融入目标背景。这两个模块在各自的任务上均超越了基线模型，使得头部交换达到了业界领先水平。我们还处理了复杂情况，如源图像与目标图像在发型上存在显著差异的情形。相关代码已发布于https://github.com/ai-forever/ghost-2.0。

English

While the task of face swapping has recently gained attention in the research community, a related problem of head swapping remains largely unexplored. In addition to skin color transfer, head swap poses extra challenges, such as the need to preserve structural information of the whole head during synthesis and inpaint gaps between swapped head and background. In this paper, we address these concerns with GHOST 2.0, which consists of two problem-specific modules. First, we introduce enhanced Aligner model for head reenactment, which preserves identity information at multiple scales and is robust to extreme pose variations. Secondly, we use a Blender module that seamlessly integrates the reenacted head into the target background by transferring skin color and inpainting mismatched regions. Both modules outperform the baselines on the corresponding tasks, allowing to achieve state of the art results in head swapping. We also tackle complex cases, such as large difference in hair styles of source and target. Code is available at https://github.com/ai-forever/ghost-2.0

GHOST 2.0：生成式高保真一次性头部迁移

GHOST 2.0: generative high-fidelity one shot transfer of heads

摘要

Summary

Support