Dialog2Flow：預訓練軟對比動作驅動句子嵌入，用於自動對話流提取。

Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extraction

October 24, 2024

作者: Sergio Burdisso, Srikanth Madikeri, Petr Motlicek

cs.AI

摘要

從未標註的對話中有效地推導結構化工作流程在計算語言學中仍然是一個未被充分探討且艱鉅的挑戰。自動化這個過程可以顯著加速在新領域中手動設計工作流程，並實現大型語言模型在特定領域流程圖中的基礎，增強透明度和可控性。在本文中，我們介紹了Dialog2Flow（D2F）嵌入，它與傳統的句子嵌入不同，通過將話語映射到潛在空間，根據其交際和信息功能（即它們代表的動作）對其進行分組。D2F允許將對話建模為潛在空間中的連續軌跡，其中包含不同的與動作相關的區域。通過對D2F嵌入進行聚類，潛在空間被量化，對話可以轉換為區域/動作ID序列，從而促進對潛在工作流程的提取。為了預先訓練D2F，我們通過統一二十個任務導向對話數據集並標準化每輪動作標註，構建了一個全面的數據集。我們還引入了一種新穎的軟對比損失，利用這些動作的語義信息來引導表示學習過程，顯示出優於標準監督對比損失的性能。通過與各種句子嵌入進行評估，包括對話特定的嵌入，證明了D2F在各種領域中產生出優越的定性和定量結果。

English

Efficiently deriving structured workflows from unannotated dialogs remains an underexplored and formidable challenge in computational linguistics. Automating this process could significantly accelerate the manual design of workflows in new domains and enable the grounding of large language models in domain-specific flowcharts, enhancing transparency and controllability. In this paper, we introduce Dialog2Flow (D2F) embeddings, which differ from conventional sentence embeddings by mapping utterances to a latent space where they are grouped according to their communicative and informative functions (i.e., the actions they represent). D2F allows for modeling dialogs as continuous trajectories in a latent space with distinct action-related regions. By clustering D2F embeddings, the latent space is quantized, and dialogs can be converted into sequences of region/action IDs, facilitating the extraction of the underlying workflow. To pre-train D2F, we build a comprehensive dataset by unifying twenty task-oriented dialog datasets with normalized per-turn action annotations. We also introduce a novel soft contrastive loss that leverages the semantic information of these actions to guide the representation learning process, showing superior performance compared to standard supervised contrastive loss. Evaluation against various sentence embeddings, including dialog-specific ones, demonstrates that D2F yields superior qualitative and quantitative results across diverse domains.