SpotLight：透過擴散實現陰影引導的物體照明

摘要

最近的研究表明擴散模型可以作為強大的神經渲染引擎，可用於將虛擬物體插入圖像。然而，與典型基於物理的渲染器不同，神經渲染引擎受到對照明設置的手動控制不足的限制，這通常對於改善或個性化所需的圖像結果至關重要。在本文中，我們展示了對於物體照明控制的精確性可以通過簡單指定物體的期望陰影來實現。令人驚訝的是，我們發現只需將物體的陰影注入預先訓練的基於擴散的神經渲染器，即可使其根據期望的光源位置準確著色物體，同時將物體（及其陰影）與目標背景圖像協調一致。我們的方法SpotLight 利用現有的神經渲染方法，實現了可控的照明效果，而無需額外訓練。具體來說，我們展示了它與近期文獻中的兩個神經渲染器的應用。我們展示了SpotLight在物體合成結果方面取得了優越的表現，無論是從量化還是感知上，都經由用戶研究確認，勝過了專門設計用於照明的現有基於擴散的模型。

English

Recent work has shown that diffusion models can be used as powerful neural rendering engines that can be leveraged for inserting virtual objects into images. Unlike typical physics-based renderers, however, neural rendering engines are limited by the lack of manual control over the lighting setup, which is often essential for improving or personalizing the desired image outcome. In this paper, we show that precise lighting control can be achieved for object relighting simply by specifying the desired shadows of the object. Rather surprisingly, we show that injecting only the shadow of the object into a pre-trained diffusion-based neural renderer enables it to accurately shade the object according to the desired light position, while properly harmonizing the object (and its shadow) within the target background image. Our method, SpotLight, leverages existing neural rendering approaches and achieves controllable relighting results with no additional training. Specifically, we demonstrate its use with two neural renderers from the recent literature. We show that SpotLight achieves superior object compositing results, both quantitatively and perceptually, as confirmed by a user study, outperforming existing diffusion-based models specifically designed for relighting.

SpotLight：透過擴散實現陰影引導的物體照明

SpotLight: Shadow-Guided Object Relighting via Diffusion

摘要

Summary

Support