MapQaTor：一个用于高效标注地图查询数据集的系统

MapQaTor: A System for Efficient Annotation of Map Query Datasets

December 30, 2024

作者: Mahir Labib Dihan, Mohammed Eunus Ali, Md Rizwan Parvez

cs.AI

摘要

像Google地图、Apple地图、OpenStreet地图这样的映射和导航服务对于访问各种基于位置的数据至关重要，但它们通常难以处理自然语言地理空间查询。最近大型语言模型（LLMs）的进展显示了在问答（QA）方面的潜力，但从地图服务中创建可靠的地理空间QA数据集仍具有挑战性。我们介绍了MapQaTor，这是一个简化地图为基础的QA数据集创建过程的网络应用程序。通过其即插即用的架构，MapQaTor可以与任何地图API轻松集成，允许用户以最小的设置收集和可视化来自不同来源的数据。通过缓存API响应，该平台确保了一致的地面真实性，增强了数据的可靠性，即使在现实世界信息不断发展的情况下也是如此。MapQaTor将数据检索、注释和可视化集中在一个平台内，为评估基于LLM的地理空间推理的当前状态提供了独特机会，同时推动其能力以改进地理空间理解。评估指标显示，与手动方法相比，MapQaTor至少可以加快注释过程30倍，突显了其开发地理空间资源（如复杂地图推理数据集）的潜力。该网站已上线，网址为：https://mapqator.github.io/，同时提供演示视频：https://youtu.be/7_aV9Wmhs6Q。

English

Mapping and navigation services like Google Maps, Apple Maps, Openstreet Maps, are essential for accessing various location-based data, yet they often struggle to handle natural language geospatial queries. Recent advancements in Large Language Models (LLMs) show promise in question answering (QA), but creating reliable geospatial QA datasets from map services remains challenging. We introduce MapQaTor, a web application that streamlines the creation of reproducible, traceable map-based QA datasets. With its plug-and-play architecture, MapQaTor enables seamless integration with any maps API, allowing users to gather and visualize data from diverse sources with minimal setup. By caching API responses, the platform ensures consistent ground truth, enhancing the reliability of the data even as real-world information evolves. MapQaTor centralizes data retrieval, annotation, and visualization within a single platform, offering a unique opportunity to evaluate the current state of LLM-based geospatial reasoning while advancing their capabilities for improved geospatial understanding. Evaluation metrics show that, MapQaTor speeds up the annotation process by at least 30 times compared to manual methods, underscoring its potential for developing geospatial resources, such as complex map reasoning datasets. The website is live at: https://mapqator.github.io/ and a demo video is available at: https://youtu.be/7_aV9Wmhs6Q.