Rapid Classification of Humanitarian Information with Lightweight LLMs

Timely classification of humanitarian information from social media is critical for effective disaster response. This paper presents a lightweight, cost-effective framework for disaster tweet classification using parameter-efficient fine-tuning.

Implementation Details

The framework was evaluated on a unified corpus based on the HumAID dataset (76,484 tweets across 19 disaster events). The approach relies on fine-tuning Llama 3.1 8B using LoRA (Low-Rank Adaptation). The results show that LoRA achieves 79.62% accuracy in humanitarian classification, with training of only ~2% of the parameters. The use of QLoRA further reduces memory costs, maintaining 99.4% of LoRA performance.

RAG and Label Noise

Contrary to common assumptions, RAG (Retrieval-Augmented Generation) strategies degrade fine-tuned model performance due to label noise from retrieved examples. This study establishes a practical, reproducible pipeline for building reliable crisis intelligence systems with limited computational resources. For those evaluating on-premise deployments, there are trade-offs that AI-RADAR analyzes in detail at /llm-onpremise.