BlueLM-V-3B：移动设备上多模态大型语言模型的算法与系统协同设计

BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices

November 16, 2024

作者: Xudong Lu, Yinghao Chen, Cheng Chen, Hui Tan, Boheng Chen, Yina Xie, Rui Hu, Guanxin Tan, Renshou Wu, Yan Hu, Yi Zeng, Lei Wu, Liuyang Bian, Zhaoxiong Wang, Long Liu, Yanzhou Yang, Han Xiao, Aojun Zhou, Yafei Wen, Xiaoxin Chen, Shuai Ren, Hongsheng Li

cs.AI

摘要

多模态大型语言模型（MLLMs）的出现和日益普及具有显著潜力，可以增强日常生活的各个方面，从改善沟通到促进学习和问题解决。作为必不可少的日常伴侣，手机代表了最有效和可访问的部署平台，可实现MLLMs的无缝集成到日常任务中。然而，在手机上部署MLLMs面临挑战，因为内存大小和计算能力有限，这使得在没有广泛优化的情况下难以实现平滑和实时处理。在本文中，我们提出了BlueLM-V-3B，这是一种专门针对在移动平台上高效部署MLLMs的算法和系统共同设计方法。具体来说，我们重新设计了主流MLLMs采用的动态分辨率方案，并实施了硬件感知部署的系统优化，以优化在手机上的模型推断。BlueLM-V-3B具有以下主要亮点：（1）体积小：BlueLM-V-3B具有包含27亿参数的语言模型和4亿参数的视觉编码器。（2）速度快：BlueLM-V-3B在联发科Dimensity 9300处理器上实现了24.4个标记/秒的生成速度，采用了4位LLM权重量化。（3）性能强：BlueLM-V-3B在OpenCompass基准测试中取得了66.1的最高平均分，超过了一系列参数规模更大的模型（例如MiniCPM-V-2.6，InternVL2-8B）。

English

The emergence and growing popularity of multimodal large language models (MLLMs) have significant potential to enhance various aspects of daily life, from improving communication to facilitating learning and problem-solving. Mobile phones, as essential daily companions, represent the most effective and accessible deployment platform for MLLMs, enabling seamless integration into everyday tasks. However, deploying MLLMs on mobile phones presents challenges due to limitations in memory size and computational capability, making it difficult to achieve smooth and real-time processing without extensive optimization. In this paper, we present BlueLM-V-3B, an algorithm and system co-design approach specifically tailored for the efficient deployment of MLLMs on mobile platforms. To be specific, we redesign the dynamic resolution scheme adopted by mainstream MLLMs and implement system optimization for hardware-aware deployment to optimize model inference on mobile phones. BlueLM-V-3B boasts the following key highlights: (1) Small Size: BlueLM-V-3B features a language model with 2.7B parameters and a vision encoder with 400M parameters. (2) Fast Speed: BlueLM-V-3B achieves a generation speed of 24.4 token/s on the MediaTek Dimensity 9300 processor with 4-bit LLM weight quantization. (3) Strong Performance: BlueLM-V-3B has attained the highest average score of 66.1 on the OpenCompass benchmark among models with leq 4B parameters and surpassed a series of models with much larger parameter sizes (e.g., MiniCPM-V-2.6, InternVL2-8B).