AfriEconQA: A New Benchmark for African Economic Analysis

AfriEconQA, a specialized benchmark dataset for African economic analysis, has been released. The dataset is based on a corpus of 236 World Bank reports and aims to evaluate the ability of Information Retrieval (IR) and Retrieval-Augmented Generation (RAG) systems to answer complex economic queries.

Dataset Details

AfriEconQA consists of 8,937 question-answering (QA) instances carefully selected from a pool of over 10,000 synthetic questions. Each instance includes:

  • A question requiring reasoning about economic indicators.
  • The corresponding evidence extracted from the reports.
  • A verified answer.
  • Source metadata (URL, publication date).

Challenges and Objectives

AfriEconQA represents a significant challenge for IR systems, as the data is largely absent from the pre-training corpora of current Large Language Models (LLMs). Initial experiments, which compared a zero-shot model (GPT-5 Mini) with RAG configurations based on GPT-4o and Qwen 32B, highlighted significant gaps in parametric knowledge. Zero-shot models failed to answer more than 90% of the questions, and even the most advanced RAG pipelines struggled to achieve high precision. The dataset and code will be made publicly available upon publication.