MapQaTor：一個用於高效標註地圖查詢數據集的系統

MapQaTor: A System for Efficient Annotation of Map Query Datasets

December 30, 2024

作者: Mahir Labib Dihan, Mohammed Eunus Ali, Md Rizwan Parvez

cs.AI

摘要

像Google Maps、Apple Maps、OpenStreet Maps等的地圖和導航服務對於訪問各種基於位置的數據至關重要，但它們通常難以處理自然語言的地理空間查詢。最近大型語言模型（LLMs）的進展在問答（QA）方面表現出潛力，但從地圖服務中創建可靠的地理空間QA數據集仍然具有挑戰性。我們介紹了MapQaTor，這是一個網絡應用程序，可簡化可重現、可追踪的基於地圖的QA數據集的創建。通過其即插即用的架構，MapQaTor實現了與任何地圖API的無縫集成，使用戶可以在最小設置的情況下從各種來源收集和可視化數據。通過緩存API響應，該平台確保了一致的基本事實，增強了數據的可靠性，即使在現實世界信息演變的情況下也是如此。MapQaTor將數據檢索、標註和可視化集中在一個平台內，為評估基於LLM的地理空間推理的當前狀態提供了獨特機會，同時推動其能力以改進地理空間理解。評估指標顯示，與手動方法相比，MapQaTor至少可以將標註過程加快30倍，突顯了其發展複雜地圖推理數據集等地理空間資源的潛力。該網站可通過以下鏈接訪問：https://mapqator.github.io/，演示視頻可在以下鏈接中觀看：https://youtu.be/7_aV9Wmhs6Q。

English

Mapping and navigation services like Google Maps, Apple Maps, Openstreet Maps, are essential for accessing various location-based data, yet they often struggle to handle natural language geospatial queries. Recent advancements in Large Language Models (LLMs) show promise in question answering (QA), but creating reliable geospatial QA datasets from map services remains challenging. We introduce MapQaTor, a web application that streamlines the creation of reproducible, traceable map-based QA datasets. With its plug-and-play architecture, MapQaTor enables seamless integration with any maps API, allowing users to gather and visualize data from diverse sources with minimal setup. By caching API responses, the platform ensures consistent ground truth, enhancing the reliability of the data even as real-world information evolves. MapQaTor centralizes data retrieval, annotation, and visualization within a single platform, offering a unique opportunity to evaluate the current state of LLM-based geospatial reasoning while advancing their capabilities for improved geospatial understanding. Evaluation metrics show that, MapQaTor speeds up the annotation process by at least 30 times compared to manual methods, underscoring its potential for developing geospatial resources, such as complex map reasoning datasets. The website is live at: https://mapqator.github.io/ and a demo video is available at: https://youtu.be/7_aV9Wmhs6Q.