從單一角色素描生成的3D動畫

DrawingSpinUp: 3D Animation from Single Character Drawings

September 13, 2024

作者: Jie Zhou, Chufeng Xiao, Miu-Ling Lam, Hongbo Fu

cs.AI

摘要

將各種角色素描進行動畫化是一項引人入勝的視覺內容創作任務。對於單一角色素描，現有的動畫方法僅限於平面2D運動，因此缺乏3D效果。另一種解決方案是從角色素描中重建3D模型作為代理，然後將3D運動數據重新定位到該模型上。然而，現有的圖像到3D的方法在外觀和幾何方面對業餘角色素描效果不佳。我們觀察到角色素描中常見的輪廓線會因其視角依賴性而在紋理合成中引入顯著的歧義。此外，由單線輪廓表示的細小區域（例如棒人的細肢）由於其精細結構而難以重建。為了應對這些問題，我們提出了一個新穎的系統，名為DrawingSpinUp，以產生可信的3D動畫，賦予角色素描生命力，使其可以自由旋轉、跳躍，甚至進行嘻哈舞蹈。為了改善外觀，我們採用了一種先刪除再恢復的策略，首先刪除視角依賴的輪廓線，然後在重新定位重建的角色後再渲染它們。為了改進幾何結構，我們開發了一種基於骨架的細化變形算法，以完善由單線輪廓表示的細小結構。實驗評估和感知用戶研究表明，我們提出的方法優於現有的2D和3D動畫方法，並從單一角色素描生成高質量的3D動畫。請參考我們的項目頁面（https://lordliang.github.io/DrawingSpinUp）以獲取代碼和生成的動畫。

English

Animating various character drawings is an engaging visual content creation task. Given a single character drawing, existing animation methods are limited to flat 2D motions and thus lack 3D effects. An alternative solution is to reconstruct a 3D model from a character drawing as a proxy and then retarget 3D motion data onto it. However, the existing image-to-3D methods could not work well for amateur character drawings in terms of appearance and geometry. We observe the contour lines, commonly existing in character drawings, would introduce significant ambiguity in texture synthesis due to their view-dependence. Additionally, thin regions represented by single-line contours are difficult to reconstruct (e.g., slim limbs of a stick figure) due to their delicate structures. To address these issues, we propose a novel system, DrawingSpinUp, to produce plausible 3D animations and breathe life into character drawings, allowing them to freely spin up, leap, and even perform a hip-hop dance. For appearance improvement, we adopt a removal-then-restoration strategy to first remove the view-dependent contour lines and then render them back after retargeting the reconstructed character. For geometry refinement, we develop a skeleton-based thinning deformation algorithm to refine the slim structures represented by the single-line contours. The experimental evaluations and a perceptual user study show that our proposed method outperforms the existing 2D and 3D animation methods and generates high-quality 3D animations from a single character drawing. Please refer to our project page (https://lordliang.github.io/DrawingSpinUp) for the code and generated animations.