OCR models: Tesseract, PaddleOCR-VL, or a document VLM?
Choose an optical character recognition (OCR) system from the document and the required output. Clean printed pages, forms, tables, receipts, research papers, screenshots, and handwriting pose different problems. Decide first whether you need raw text, page layout, structured fields, or answers supported by the document.
For a PDF, first check whether its embedded text and layout are usable. Extract them directly if they meet the task. Test classical OCR such as Tesseract for clean printed scans. Test PaddleOCR-VL 1.6 for self-hosted layout, text, tables, and formulas. Test dots.mocr for structured graphics. A general vision-language model (VLM) is another candidate when the task also requires visual question answering.
Compare candidates on your document types, required fields, data controls, latency, and cost.
Decision table
| Document problem | Starting point | Why |
|---|---|---|
| PDF with usable embedded text | PDF text/layout extraction | Avoid recognizing text already stored in the file. |
| Clean printed scans at large volume | Classical OCR pipeline | Cheap, predictable, CPU-friendly, easy to batch. |
| Complex PDFs with tables, formulas, and figures | PaddleOCR-VL 1.6 or a hosted document VLM | Test a specialized parser against a general model on the same pages. |
| Structured field extraction | VLM plus structured output, or a domain parser | The output schema matters as much as text recognition. |
| Data-sensitive documents | Self-hosted OCR or open VLM | Keeps documents inside your environment. |
| Layout reconstruction | PaddleOCR-VL, dots.mocr, or another layout parser | Plain OCR text loses reading order, tables, captions, and sections. |
| Human review workflows | OCR plus confidence and span provenance | Reviewers need page, box, field, and source traceability. |
OCR and document understanding
Traditional OCR extracts characters. Document AI also needs layout and meaning. Table cells, checkboxes, signatures, captions, and footnotes may all contain text, but flattening them into one string destroys their relationships. That loss then breaks field extraction and question answering.
Vision-language models can extract fields and answer questions about diagrams or tables. Their output still needs checks against the page. PaddleOCR-VL 1.6 combines layout analysis with a 0.9B VLM component, while dots.mocr also parses structured graphics. Compare those pipelines with classical OCR on the pages and fields you need; a larger model is not a reason to replace a working extractor.
Model and tool classes
| Class | Strength | Weakness | Use it when |
|---|---|---|---|
| Tesseract-style OCR | Cost, transparency, offline execution | Weak on handwriting, layout, noisy scans, and rich documents | The input is clean printed text and the task is text extraction. |
| Cloud document VLM | Handles complex PDFs, charts, tables, and multimodal context | Cost, latency, data residency, API dependency | You need high-quality extraction or QA over varied documents. |
| PaddleOCR-VL 1.6 | Text, tables, formulas, charts, layout, and 109 languages | Full pipeline is more complex than its VLM component alone | You need a maintained self-hosted document parser. |
| dots.mocr | Document text plus structured-graphics parsing | 3B serving footprint and task-dependent output quality | Charts, figures, or SVG-oriented reconstruction matter. |
| Open general VLM | Data control, customization, and broader visual reasoning | Serving complexity and model variance | You need self-hosting beyond document parsing. |
| Layout parser | Boxes, reading order, document structure | Usually needs orchestration with OCR or VLM | Layout fidelity matters. |
| Hybrid pipeline | Cost control and routing | More engineering | You can route easy pages to cheap OCR and hard pages to VLMs. |
Practical architecture
For production document AI, I would not send every page to the most expensive model by default.
- Inspect the file and record page-level metadata. For PDFs, check whether existing text and reading order meet the task.
- Extract usable embedded text directly. PyMuPDF recommends checking whether OCR is needed before running it on a page.
- Route clean printed scans without usable text to classical OCR.
- Route tables, low-confidence pages, handwritten regions, and complex layouts to a VLM or specialized parser.
- Require structured output for fields, then validate values against the page. A schema can enforce field shape without proving that a value appears in the document.
- Store source spans, page numbers, bounding boxes where available, and model version.
- Sample human review by confidence, document type, and downstream impact.
Add routing when testing shows that different page types need different methods. Check that the router does not send a difficult page down a path that loses required fields.
Evaluation checklist
Do not evaluate OCR only with character error rate. For document AI, track:
- field-level accuracy for extracted fields
- table cell preservation
- reading-order accuracy
- page coverage
- unsupported field rate
- citation or span support for extracted claims
- human correction rate
- cost per page and latency per document
Deeper reading
- OCR in 2026 covers pipelines, VLMs, layout, evaluation, and cost.
- RAG Evaluation Metrics is relevant when OCR feeds a retrieval system.