大型语言模型（LLMs）中的开源优势

The Open Source Advantage in Large Language Models (LLMs)

December 16, 2024

作者: Jiya Manchanda, Laura Boettcher, Matheus Westphalen, Jasser Jasser

cs.AI

摘要

大型语言模型（LLMs）标志着自然语言处理（NLP）中的关键转变，已经在文本生成、翻译和领域特定推理方面取得了进展。像 GPT-4 这样的闭源模型，由专有数据集和大量计算资源驱动，如今在性能上处于领先地位。然而，它们因其“黑匣子”性质以及以一种阻碍可重现性和公平AI发展的方式限制可访问性而受到批评。相比之下，像 LLaMA 和 BLOOM 这样的开源倡议通过社区驱动的开发和计算效率优先考虑民主化。这些模型在减小性能差距方面取得了显著进展，特别是在语言多样性和领域特定应用方面，同时为全球研究人员和开发者提供了可访问的工具。值得注意的是，这两种范式都依赖于基础架构创新，比如 Vaswani 等人（2017）提出的 Transformer 框架。闭源模型通过有效扩展规模而表现出色，而开源模型则适应了未被充分代表的语言和领域的实际应用。像低秩适应（LoRA）和指导调整数据集这样的技术使开源模型在资源有限的情况下取得了竞争性结果。可以肯定的是，闭源和开源方法之间的紧张关系突显了AI中透明度与专有控制的更广泛辩论。伦理考虑进一步凸显了这种分歧。闭源系统限制了外部审查，而开源模型促进了可重现性和协作，但缺乏标准化的审计文档框架来减轻偏见。利用两种范式优势的混合方法可能会塑造LLM创新的未来，确保可访问性、竞争性技术性能和道德部署。

English

Large language models (LLMs) mark a key shift in natural language processing (NLP), having advanced text generation, translation, and domain-specific reasoning. Closed-source models like GPT-4, powered by proprietary datasets and extensive computational resources, lead with state-of-the-art performance today. However, they face criticism for their "black box" nature and for limiting accessibility in a manner that hinders reproducibility and equitable AI development. By contrast, open-source initiatives like LLaMA and BLOOM prioritize democratization through community-driven development and computational efficiency. These models have significantly reduced performance gaps, particularly in linguistic diversity and domain-specific applications, while providing accessible tools for global researchers and developers. Notably, both paradigms rely on foundational architectural innovations, such as the Transformer framework by Vaswani et al. (2017). Closed-source models excel by scaling effectively, while open-source models adapt to real-world applications in underrepresented languages and domains. Techniques like Low-Rank Adaptation (LoRA) and instruction-tuning datasets enable open-source models to achieve competitive results despite limited resources. To be sure, the tension between closed-source and open-source approaches underscores a broader debate on transparency versus proprietary control in AI. Ethical considerations further highlight this divide. Closed-source systems restrict external scrutiny, while open-source models promote reproducibility and collaboration but lack standardized auditing documentation frameworks to mitigate biases. Hybrid approaches that leverage the strengths of both paradigms are likely to shape the future of LLM innovation, ensuring accessibility, competitive technical performance, and ethical deployment.