Back to all available

Traffic document processing with large language models: A benchmark for information extraction from noisy OCR

artificial-intelligence
computer-vision
data-mining-and-machine-learning
natural-language-and-speech
software-engineering

Abstract

The accurate extraction of structured information from scanned administrative documents remains a key challenge in traffic management systems and logistics workflows. This study presents a hybrid AI pipeline that integrates Optical Character Recognition (OCR) and Large Language Models (LLMs) for robust information extraction from noisy, semi-structured documents. While prior studies often treat OCR and language understanding as separate stages, we jointly evaluate them across multiple OCR engines (Tesseract, PaddleOCR) and instruction-tuned LLMs (e.g., Mistral, LLaMA) on real-world documents such as rate confirmations and commercial driver licenses. Our approach addresses the critical but underexplored problem of interpreting fragmented and layout-free OCR text using LLMs, proposing tailored prompt strategies to maximize accuracy without relying on template-based rules. We show that specific LLMs exhibit resilience to OCR noise and can infer document semantics with minimal supervision. Results show that PaddleOCR provides more complete text at a higher cost, Mistral 7B with JSON-style prompting yields the most reliable structured outputs, and vision grounding reduces hallucinations while improving the extraction of location and time fields. By releasing code, annotations, and evaluation scripts, this work establishes a reproducible benchmark for OCR–LLM pipelines, highlighting both their limitations and potential in logistics, compliance, and back-office automation.
Ask to review this manuscript

Notes for potential reviewers

  • Volunteering is not a guarantee that you will be asked to review. There are many reasons: reviewers must be qualified, there should be no conflicts of interest, a minimum of two reviewers have already accepted an invitation, etc.
  • This is NOT OPEN peer review. The review is single-blind, and all recommendations are sent privately to the Academic Editor handling the manuscript. All reviews are published and reviewers can choose to sign their reviews.
  • What happens after volunteering? It may be a few days before you receive an invitation to review with further instructions. You will need to accept the invitation to then become an official referee for the manuscript. If you do not receive an invitation it is for one of many possible reasons as noted above.
  • PeerJ Computer Science does not judge submissions based on subjective measures such as novelty, impact or degree of advance. Effectively, reviewers are asked to comment on whether or not the submission is scientifically and technically sound and therefore deserves to join the scientific literature. Our Peer Review criteria can be found on the "Editorial Criteria" page - reviewers are specifically asked to comment on 3 broad areas: "Basic Reporting", "Experimental Design" and "Validity of the Findings".
  • Reviewers are expected to comment in a timely, professional, and constructive manner.
  • Until the article is published, reviewers must regard all information relating to the submission as strictly confidential.
  • When submitting a review, reviewers are given the option to "sign" their review (i.e. to associate their name with their comments). Otherwise, all review comments remain anonymous.
  • All reviews of published articles are published. This includes manuscript files, peer review comments, author rebuttals and revised materials.
  • Each time a decision is made by the Academic Editor, each reviewer will receive a copy of the Decision Letter (which will include the comments of all reviewers).

If you have any questions about submitting your review, please email us at [email protected].