音频匹配切割：在电影和视频中查找和创建匹配的音频过渡

Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos

August 20, 2024

作者: Dennis Fedorishin, Lie Lu, Srirangaraj Setlur, Venu Govindaraju

cs.AI

摘要

"匹配剪辑"是一种常见的视频编辑技术，其中一对具有相似构图的镜头之间可以流畅地过渡。虽然匹配剪辑通常是视觉上的，但某些匹配剪辑涉及音频的流畅过渡，其中来自不同来源的声音融合为一个无法区分的过渡，连接两个镜头。在本文中，我们探讨了自动查找和创建视频和电影中的"音频匹配剪辑"的能力。我们为音频匹配剪辑创建了一种自监督音频表示，并开发了一个粗到精的音频匹配流程，推荐匹配镜头并创建混合音频。我们进一步为拟议的音频匹配剪辑任务注释了一个数据集，并比较了多种音频表示的能力，以找到音频匹配剪辑候选项。最后，我们评估了多种方法来混合两个匹配的音频候选项，以实现平滑过渡。项目页面和示例可在以下链接找到：https://denfed.github.io/audiomatchcut/

English

A "match cut" is a common video editing technique where a pair of shots that have a similar composition transition fluidly from one to another. Although match cuts are often visual, certain match cuts involve the fluid transition of audio, where sounds from different sources merge into one indistinguishable transition between two shots. In this paper, we explore the ability to automatically find and create "audio match cuts" within videos and movies. We create a self-supervised audio representation for audio match cutting and develop a coarse-to-fine audio match pipeline that recommends matching shots and creates the blended audio. We further annotate a dataset for the proposed audio match cut task and compare the ability of multiple audio representations to find audio match cut candidates. Finally, we evaluate multiple methods to blend two matching audio candidates with the goal of creating a smooth transition. Project page and examples are available at: https://denfed.github.io/audiomatchcut/