Fact-checking Enhanced by LLMs and Knowledge Graphs

The spread of misinformation online poses a significant threat. A new study introduces a method for fact-checking that leverages LLMs and open knowledge graphs to retrieve accurate and reliable evidence.

Traditional methods rely on semantic and contextual patterns learned from training data, limiting their generalization. Retrieval Augmented Generation (RAG) based techniques utilize the reasoning capabilities of LLMs with retrieved evidence documents, but often rely on textual similarity, overlooking more complex factual correlations.

WKGFC: A New Approach

The new approach, called WKGFC, uses an authorized knowledge graph as a core resource of evidence. An LLM assesses claims and retrieves the most relevant knowledge subgraphs, forming structured evidence for fact verification. To complete the knowledge graph evidence, web contents are retrieved.

This process is implemented as an automatic Markov Decision Process (MDP). An LLM agent decides what actions to take based on the current evidence and the claims. The MDP is adapted for fact-checking through prompt optimization for the agentic LLM.

For those evaluating on-premise deployments, there are trade-offs to consider. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these aspects.