CroFinBen: A Multilingual Benchmark of Large Language Models Crossing Mainstream and Low-Resource Finance

Recent advancements in financial LLMs (FinLLMs) have shown strong performance in mainstream languages like English and Chinese, but are limited in low-resource settings，particularly in Southeast Asia (SEA), where annotated data resources are extremely scarce. Existing benchmarks primarily focus on high-resource finance, neglecting low-resource finance. To address this, we introduce CroFinBen, the first multilingual benchmark specifically designed to bridge the language-resource gap between mainstream and low-resource finance. It includes four key financial tasks (FinSA, FinSP, FinTS, and FinTC) in both high-resource languages (English and Chinese) and low-resource SEA languages (Indonesian, Malaysian, Thai, Filipino, and Vietnamese), comprising 50k samples from 16 datasets to ensure a comprehensive and balanced evaluation. Unlike others that rely on full translations or overlook local context, CroFinBen incorporates localized annotations to reflect financial terms and cultural nuances in SEA languages. Evaluating 25 LLMs shows significant performance differences, with no clear proficiency in either mainstream or low-resource languages, especially in language-biased fine-tuned FinLLMs. Open-source 1800B large-parameter GPT-4o excels, while DeepSeek-V3 and ChatGPT-3.5 also perform well. By bridging language-resource barriers, CroFinBen enhances the fairness and robustness of FinLLMs, providing strong support for improving performance in global financial scenarios. Our resources are available at https://github.com/XinXing3/CroFinben.