The Need for Structurally Faithful OCR for LaTeX

Scientific publishing heavily relies on LaTeX for its ability to handle complex formatting, cross-references, and mathematical structures. However, existing Optical Character Recognition (OCR) systems, while excelling at extracting plain text or Markdown, often overlook the intrinsic structural and executable properties of LaTeX documents. This limitation prevents a faithful reconstruction of scientific PDFs, rendering the results unusable for further editing or compilation.

The TexOCR project addresses this gap by proposing an innovative approach for page-level reconstruction of scientific PDFs into compilable LaTeX. The goal is to overcome current limitations, ensuring that the conversion is not merely textual but also maintains the structural integrity and compilability of the original document.

TexOCR: Architecture and Training Methodology

At the core of the TexOCR project is a 2-billion-parameter model, trained using a combination of Supervised Fine-tuning (SFT) and Reinforcement Learning (RL). A distinctive element is the use of "verifiable rewards" derived from LaTeX unit tests, which directly enforce the compilability and referential integrity of the generated code. This approach is crucial for ensuring that the output is not only syntactically correct but also functionally valid.

To support development and evaluation, the team introduced TexOCR-Bench, a multi-dimensional benchmark, and TexOCR-Train, a vast training corpus. TexOCR-Bench is designed to jointly assess transcription fidelity, structural faithfulness, and end-to-end compilability. Tests conducted on 21 frontier models revealed that existing systems frequently violate key document invariants, such as consistent section structure, correct float placement, and valid label-reference links. These violations undermine compilation reliability and downstream usability of the converted documents.

Implications and Advantages of Reinforcement Learning

The analysis of the results highlighted that Reinforcement Learning, particularly with the implementation of verifiable rewards, offers consistent improvements over Supervised Fine-tuning alone. These advancements are particularly evident in metrics related to structure and compilation, areas where traditional systems show the greatest shortcomings. This suggests that RL is a powerful tool for instilling in Large Language Models the ability to adhere to complex and verifiable constraints, essential for critical applications like technical document conversion.

For organizations considering on-premise LLM deployment, a model's ability to produce highly structured and verifiable outputs is fundamental. Data fidelity and structural compliance are often indispensable requirements, especially in regulated sectors. TexOCR's methodology, which emphasizes verifiability and integrity, aligns well with the control and data sovereignty needs typical of self-hosted environments.

Future Prospects for Document Automation

TexOCR's work represents a significant step towards advanced automation of scientific document management. The ability to convert PDFs into compilable LaTeX with high structural fidelity opens new possibilities for indexing, reprocessing, and accessibility of scientific content. This can not only reduce manual workload but also improve the quality and consistency of digital documents.

In a broader context, the research demonstrates the potential of Large Language Models, when appropriately trained with rigorous verification mechanisms, to address complex challenges that go beyond simple text generation. The emphasis on compilability and referential integrity sets a new standard for structured document OCR, offering valuable insights for the development of more reliable and precise AI systems across various application domains.