VisualLens：透過視覺歷史的個人化

VisualLens: Personalization through Visual History

November 25, 2024

作者: Wang Bill Zhu, Deqing Fu, Kai Sun, Yi Lu, Zhaojiang Lin, Seungwhan Moon, Kanika Narang, Mustafa Canim, Yue Liu, Anuj Kumar, Xin Luna Dong

cs.AI

摘要

我們假設使用者在影像中反映日常生活的視覺歷史，提供了有價值的洞察力，能夠揭示他們的興趣和偏好，並可用於個性化。在實現這一目標時，最主要的挑戰之一是視覺歷史中的多樣性和噪音，其中包含不一定與推薦任務相關的圖像，也不一定反映使用者的興趣，甚至不一定與偏好相關。現有的推薦系統要麼依賴於特定任務的使用者互動日誌，例如用於購物推薦的線上購物歷史，要麼專注於文本信號。我們提出了一種新穎的方法，VisualLens，它提取、過濾和優化圖像表示，並利用這些信號進行個性化。我們創建了兩個新的基準，具有與任務無關的視覺歷史，並展示了我們的方法在Hit@3上比最先進的推薦提高了5-10%，在GPT-4o上提高了2-5%。我們的方法為傳統方法失敗的情況下的個性化推薦打開了道路。

English

We hypothesize that a user's visual history with images reflecting their daily life, offers valuable insights into their interests and preferences, and can be leveraged for personalization. Among the many challenges to achieve this goal, the foremost is the diversity and noises in the visual history, containing images not necessarily related to a recommendation task, not necessarily reflecting the user's interest, or even not necessarily preference-relevant. Existing recommendation systems either rely on task-specific user interaction logs, such as online shopping history for shopping recommendations, or focus on text signals. We propose a novel approach, VisualLens, that extracts, filters, and refines image representations, and leverages these signals for personalization. We created two new benchmarks with task-agnostic visual histories, and show that our method improves over state-of-the-art recommendations by 5-10% on Hit@3, and improves over GPT-4o by 2-5%. Our approach paves the way for personalized recommendations in scenarios where traditional methods fail.