Coder Legion - Recent posts tagged ocr

Building a Smart OCR Converter with Python: Turning Images into Editable Text

Sat, 10 Oct 2026 16:26:56 +0000

In today’s digital world, information is everywhere but not all information is available in a usable format. Many businesses still manage scanned documents, printed forms, invoices, receipts, and images containing important text. Manually typing this information into a computer takes time, increases the risk of errors, and reduces productivity.

This is where an OCR (Optical Character Recognition) converter built with Python can make a real difference.

The Problem: Text Locked Inside Images

Imagine a business receiving hundreds of scanned documents every week. Employees need to read each document, extract the information, and manually enter it into a digital system.

This process creates several challenges:

Time-consuming work: Manually copying text from documents wastes valuable hours.
Human errors: Incorrect spelling, missing information, and typing mistakes can affect data accuracy.
Difficult document management: Searching for specific information inside image-based documents is not easy.
Limited productivity: Employees spend more time processing documents and less time on important tasks.
Scalability issues: As the number of documents grows, manual processing becomes increasingly difficult.

Businesses need a smarter way to convert image-based text into digital information.

The Solution: Developing an OCR Converter Using Python

To address these challenges, we can build an OCR converter using Python. The application reads text from images and scanned documents, then converts it into machine-readable text that users can copy, search, edit, or save.

Python is an excellent choice for this project because it provides powerful libraries for image processing, text recognition, and web application development.

How Does the OCR Converter Work?

Image Upload: Users upload an image containing text, such as a JPG, PNG, or a supported scanned document.

Image Processing: The application prepares the image for recognition. Techniques such as grayscale conversion, noise reduction, and thresholding can improve text readability.

Text Recognition: An OCR engine analyzes the processed image and identifies the characters and words.

Text Extraction: The recognized text appears on the screen, allowing users to review and copy the results.

Download and Export: The extracted content can be saved as a text file or integrated into other document-processing workflows.

The goal is to make document conversion simple, accessible, and efficient.

Technologies Behind the Project

A Python-based OCR converter can use several technologies working together:

Python: Handles the main application logic and processing workflow.
Tesseract OCR: Recognizes text in images and converts it into digital text.
Pytesseract: Connects Python code with the Tesseract OCR engine.
Pillow or OpenCV: Helps process images before recognition.
Flask: Can provide a lightweight web interface where users upload images and view extracted text.

These tools make it possible to build a practical OCR application without developing a text-recognition engine from scratch.

Why Image Preprocessing Matters

One of the most important lessons in OCR development is that the quality of the input image affects the quality of the extracted text.

A clear, high-resolution image usually produces better results than a blurry or poorly lit photograph.

For improved recognition, developers can implement:

Grayscale conversion to simplify image information.
Thresholding to improve contrast between text and background.
Noise removal to reduce unwanted marks.
Image resizing to improve the visibility of small characters.
Deskewing to straighten tilted documents.

However, preprocessing methods should be selected according to the document type. Excessive processing can sometimes remove important details or reduce recognition accuracy.

Real-World Applications

A Python OCR converter can support many practical use cases.

Business Administration: Extract text from invoices, forms, and printed records.

Education: Convert scanned study materials and printed notes into searchable text.

E-commerce: Help digitize product information and paper based inventory records.

Healthcare Administration: Assist with digitizing appropriate non-sensitive administrative documents, subject to privacy and security requirements.

Digital Archiving: Make historical documents and scanned records easier to search and organize.

OCR can also become part of a larger workflow that extracts structured information, classifies documents, or transfers recognized text into a database.

Challenges Developers Should Consider

Building an OCR converter is not only about extracting text. A reliable application should also handle unexpected situations.

Poor image quality can lead to incorrect recognition.
Complex layouts, tables, and multiple columns may require additional processing.
Handwriting is generally more challenging than printed text.
Multilingual documents may require additional language data.
Large files can increase processing time and memory usage.
Uploaded documents should be handled securely, with appropriate file validation, privacy protections, and retention policies.

Developers should test the application with different image formats, document layouts, languages, and image qualities before deploying it for real-world use.

Future Improvements

An OCR converter can evolve into a more advanced document processing platform.

Possible improvements include PDF support, batch image conversion, multilingual recognition, searchable document archives, structured data extraction, and integration with databases or business management systems.

A web-based interface can also make the application accessible to users without requiring them to run Python scripts manually.

Conclusion

Building an OCR converter with Python is a practical way to apply programming skills to a real-world problem. By combining Python, image preprocessing, and OCR technology, developers can reduce manual data entry and make document information easier to access.

The most important lesson is that useful software is not simply about writing code it is about solving problems efficiently and creating a better experience for users.

At Next Big Creative, we believe technology should help businesses work smarter, improve digital processes, and turn ideas into practical solutions.

You thinking and we building.

Next Big Creative
www.nextbigcreative.com

Give Your Document OCR Pipeline a Memory

Sat, 10 Oct 2026 16:20:15 +0000

After OCR, most pipelines throw the layout away and re-discover the same form from scratch every time. textract-field-memory is a small, zero-dependency Python layer that remembers where fields sit, checks they landed in the right place, tells you which layout a document is, and warns you when a form quietly changes. It doesn't do OCR. It makes the OCR you already run a lot more useful.

Where this started

I kept running into the same annoying gap on document pipelines. You send a form through Textract (or Tesseract, or whatever), you get back field names and bounding boxes, you pull the values, and then you throw all the positional information away. The next copy of that exact same form shows up an hour later and the pipeline treats it like it's never seen anything like it before.

That bugged me for a few reasons.

Nothing checks that a field came from the right spot on the page. If "Total" gets read out of the shipping box instead of the amount box, the value still looks like a number, so it sails downstream and nobody blinks.

Nothing notices when a form changes. A vendor nudges their invoice template, fields move a centimeter, and you find out three weeks later when a downstream job starts choking on garbage.

And there's no cheap way to ask "have I seen this layout before?" So every document, familiar or not, takes the same expensive road.

So I wrote a little library that sits after OCR and keeps the memory the pipeline was throwing out.

What it is, and what it definitely isn't

It's a spatial memory layer. You hand it field names plus normalized bounding boxes (x, y, width, height between 0 and 1). It learns roughly where each field tends to appear on a given template, and from then on it can validate positions, identify which template a document belongs to, and watch for drift.

It is not an OCR engine and it never touches pixels. Something upstream has to extract the fields first. This library just remembers where they ended up and reasons about that.

Under the hood there's nothing exotic going on, which I think is a feature. Spatial scoring is intersection-over-union plus the distance between box centers. Name matching is a normalized Levenshtein distance. Template identification is Jaccard overlap of the field-name sets combined with the spatial score. No model, no training step, no network calls. It's pure standard-library Python with zero dependencies, and templates are just JSON files on disk.

A quick look at the API

from field_memory import TemplateMemory
memory = TemplateMemory(store_path="./my_templates")
# 1. Learn — feed documents after OCR extraction
memory.record(document, template_id="employment-form")
# 2. Locate — spatial field lookup with a confidence score
matches = memory.locate(new_doc, "Employee Name")
# → FieldMatch(combined_score=0.95, spatial_score=0.93, within_expected_region=True)
# 3. Identify — which known layout is this?
match = memory.identify_template(new_doc)
# → TemplateMatch(template_id="employment-form", similarity_score=0.87)
# 4. Detect drift — has the layout shifted?
drift = memory.detect_drift(new_doc, "employment-form")
# → DriftReport(is_drifting=False, overall_drift_score=0.02)

Clustering comes for free. If you call record() and leave out the template_id, it either merges the document into a template it recognizes or spins up a new one. Run that for a while and you end up with a map of every distinct layout flowing through your pipeline without ever configuring one by hand.

Why there's no LLM inside

For a layout you've already seen, answering "where's this field?" is basically a dictionary lookup. Sub-millisecond, and free. The same question routed through an LLM is half a second to two seconds and costs you per field. Paying that toll on forms you've processed a thousand times is silly.

So the design leans into being complementary instead of competitive:

def smart_extract(document, field_name):
    matches = memory.locate(document, field_name)
    if matches and matches[0].within_expected_region and matches[0].combined_score >= 0.85:
        return matches[0].key_value, "spatial"   # free, instant
    return call_llm(document, field_name), "llm"  # pay only when needed

Think of it as the cheap first pass that decides whether you even need the expensive one. The 0.85 threshold is just a starting point. Crank it up if a wrong answer is costly and you'd rather fall back to the LLM more often.

Where it actually helps

Domain	What it remembers	What it catches
Invoice processing	Where fields sit per vendor layout	Vendor template changes, wrong-position reads
Insurance claims	Field positions across many form types	Unknown forms that need routing for review
Healthcare intake	Where fields land on clinic forms	Layout revisions before they break downstream
Manufacturing QA	Component positions on boards	Parts that shifted from baseline

The only real requirement is that your data has named things with bounding boxes. That's it.

Some numbers, and the caveats that go with them

I put it through a stress test using real Textract (FORMS) on generated PDFs plus scanned, deliberately degraded versions of the same documents.

Layout identification came out at 100% across five vendors whose invoices look nothing alike, trained on just three documents each. On dense tax returns with 33 fields crammed into a tight three-column grid, roughly 0.03 units apart, it located every field. Drift detection behaved the way I hoped: as I pushed forms further off their baseline, the drift scores climbed smoothly from 0.012 up to 0.047 and tripped the flag once they crossed the threshold. On injected anomalies (shifted, scattered, reversed, wrong-form) it caught four of five. And because it's all local computation, lookups stay under a millisecond.

Now the honest part.

The benchmark runs on synthetic PDFs generated with ReportLab and their scanned variants, not a big real-world labeled corpus. Sample sizes are small, around 20 forms per scenario.

There's no proper precision/recall/F1 yet because I don't have a ground-truth label set built. The numbers above are operational rather than a formal comparison against baselines, and I'd treat them that way.

The one anomaly it missed was the "same layout, just fewer fields" case. The fields that remain are still sitting in the right places, so spatially nothing looks wrong. You could argue that's correct behavior, but it's worth knowing going in.

Handwritten forms are a dead end, because Textract FORMS gives back nothing to learn from. That's an upstream limitation, not something this library can paper over.

I'd rather you hit these on the page than in production.

Try it

Clone it and install from source. This works right now, nothing to sign up for:

git clone https://github.com/aws-samples/sample-textract-field-memory
cd sample-textract-field-memory
pip install -e .
python examples/demo.py   # runs on synthetic data, no AWS needed

demo.py walks through template learning, spatial matching, anomaly detection, and drift analysis on synthetic data, so you can watch the whole thing work without credentials or config.

Repo: github.com/aws-samples/sample-textract-field-memory

It lives under aws-samples but there's no AWS dependency in it. Pure standard-library Python, works completely offline with any OCR source that gives you field names and boxes. MIT-0 licensed.

If you run document pipelines, the thing I'd most like feedback on is turning these stress tests into a real labeled benchmark. And if the "remember it, stop re-discovering it" idea resonates, you've got the whole pitch.

Bento Scan AI Document Scanner

Thu, 01 Oct 2026 01:33:38 +0000

A white-label scanner that works offline first

Bento Scan is a production-ready document scanner for Android and iOS built with Flutter. The mobile app handles the capture workflow on-device: it detects page boundaries, triggers automatic capture, corrects perspective, flattens curved pages, applies filters and exports multi-page PDFs.

Many scanner apps depend on proprietary cloud services for basic processing. Bento Scan keeps the essential scan pipeline offline, then pairs it with an optional self-hosted backend for teams that want synchronization, search and administration under their own control.

Included capabilities

Offline document-edge detection and automatic capture
Perspective correction and AI page flattening
Multi-page scanning, filters and PDF export
OCR in more than 100 languages
Searchable PDFs, folders, tags and full-text search
Quotas, administration statistics and S3-compatible storage
Responsive web dashboard
Editable Flutter, FastAPI and deployment source code

The package can be rebranded for a commercial scanner, an internal capture tool or a document workflow product. Developers receive the mobile app, backend, web dashboard, deployment configuration and training utilities rather than a closed SDK.

See device requirements, screenshots and the complete package on Codester.

DocBento AI Document Management

Thu, 01 Oct 2026 01:33:30 +0000

Private document AI with page-level citations

DocBento started from a practical problem: teams often have thousands of PDFs, scans, Office files and email attachments, but finding a trustworthy answer still means opening documents one by one. Generic chat tools can summarize text, yet they frequently lose the connection between an answer and the exact source page.

DocBento is a self-hosted document management platform that combines OCR, full-text retrieval and semantic search. Its AI assistant answers questions against the organization’s own library and cites the pages it used, so reviewers can verify the result instead of trusting an unsupported summary.

Core capabilities

PaddleOCR, Tesseract and optional cloud OCR engines
Hybrid full-text and semantic document search
AI answers with page-level citations
Folders, tags, approvals and document expiry reminders
Granular roles, restricted folders and audit logs
Support for PDF, images, Office files, email and other formats
FastAPI, React and Docker deployment for self-hosting

The platform is designed for teams that need control over data location, model providers and access policies. It can serve internal knowledge bases, compliance archives, client-document portals or review workflows. Administrators can choose offline components where privacy matters and connect external AI services only when their policies allow it.

The package includes editable source code and deployment documentation. See the complete feature list and screenshots on Codester.

Designing a Clear Upload-to-Export Flow for Four Document Jobs

Fri, 25 Sep 2026 14:56:23 +0000

An upload control can look simple while hiding a product decision: what does the person want to accomplish with the file? A scanned page, a photo of a document, searchable text, and a spreadsheet-ready receipt are related, but they are not the same job. A useful interface makes those paths legible before the user starts.

ScanPDF AI presents four entry points—Scan PDF, Photo to PDF, OCR PDF, and Receipt to Excel. The interface gives us a concrete example of organizing around outcomes rather than asking everyone to start with one generic “upload” action.

A task selector is part of the product model

Designing workflow labels means deciding which distinctions matter to users. For document work, the source file may be a PDF, a phone image, or a photo of a receipt. Yet the source alone does not tell us the desired result. Someone may want a portable document, extracted text, or structured transaction details.

Putting those options near the start lets people choose by intent. It also helps a team keep each workflow’s promise narrow. A photo-to-PDF path can focus on document conversion. An OCR path can explain recognized-text exports. A receipt path can set expectations about fields and spreadsheet output.

Show an output contract before upload

The OCR page lists PDF, JPG, PNG, and WebP inputs and describes PDF, TXT, and DOCX exports. The receipt page describes PDF, TXT, and CSV output, and names merchant, date, total, tax, currency, and line items. This is a useful pattern: tell users what kind of artifact a workflow is intended to produce before they invest time in it.

For product designers, the next improvement to look for is whether that contract remains visible after selection. Are supported formats near the file picker? Is the output type clear before a potentially lengthy upload? Does the result screen distinguish editable text from a formatted document? These are practical review questions, not assumptions about any one implementation.

Keep “recognized” separate from “verified”

OCR and extraction outputs can look authoritative because they are neatly formatted. That presentation should not imply perfect recognition. A receipt total might be confused with a subtotal; a date can be ambiguous; a low-quality image can omit a line. Products should make it easy to inspect extracted values and correct them before export.

An example on a marketing page can explain the shape of the result, but it is not evidence of performance on arbitrary files. Teams evaluating a tool should test varied samples, including skewed photos, faded print, multiple currencies, and receipts with several tax or line-item formats. Review actual outputs before incorporating them into an operational workflow.

Design for the whole handoff

The interaction does not end when a file is accepted. People need to know how the document will be processed, where the result appears, which export fits their next tool, and what they should verify. Good labels and clear output options can remove uncertainty without adding more controls.

That is the broader lesson from these four paths: map the interface to the user’s next decision. When scanning, conversion, OCR, and structured extraction are presented as distinct jobs, the product can explain each one honestly—and users can pick the outcome they actually need.

wsnap — macOS-style screen capture for Windows

Wed, 03 Jun 2026 12:42:49 +0000

Ever taken a screenshot only to fish the file out of a folder afterward — or thought you copied the image but got a file path instead? wsnap is a native Windows capture tool built around the two things people actually do with a screenshot: paste it and drag it.

Press Shift+F1 and drag a region. The instant you release the mouse, the image is on your clipboard — paste it anywhere with Ctrl+V. The floating thumbnail at the bottom-right is a real, draggable file: drop it into Slack, a chat, an editor, or an email. It stays put, so you can drop it again somewhere else.

Beyond that core:

Precise overlay — freezes the screen, brightens only your selection, shows live W×H, and a magnifier loupe with pixel coordinates and the HEX color under the cursor.
Annotation editor — arrow, shapes, pen, highlighter, text, numbered steps, mosaic/blur redaction, crop, with undo/redo.
On-device OCR (Korean + English) — PaddleOCR PP-OCRv5, fully offline, no language pack needed.
Many capture modes — region, full screen, click-to-capture a window, delayed, scrolling capture, GIF recording.
Capture history gallery, color picker (eyedropper), and filename templates.

No account, no tracking, fully offline. It runs quietly from the tray with a minimal idle footprint. GPL-3.0 open source.

Available for Windows 10/11 (x64) via Scoop, winget, or direct download.

GitHub: https://github.com/openwong2kim/wsnap

How Well Can OCR Read Doctor Handwriting in 2026?

Wed, 08 Apr 2026 06:00:04 +0000

Benchmarking four open-source OCR engines on 5,578 handwritten medical prescriptions

Key Takeaways

PP-OCRv5 (5M parameters) and GLM-OCR (0.9B parameters) both achieve 20%+ exact match on handwritten prescriptions, a 10x jump over Tesseract and EasyOCR

GLM-OCR leads on character accuracy (CER 0.328), while PP-OCRv5 leads on word accuracy (WER 0.789)

A 5M-parameter model trained on curated data rivals a 900M-parameter vision-language model

Neither engine is clinically deployable yet: even the best gets only 1 in 3 words exactly right

Last month I spent some time squinting at prescription scans, trying to figure out if a doctor wrote Amoxicillin or Amitriptyline. I got it wrong twice. That got me wondering: how would today's OCR engines handle this?

The stakes are real. Medication errors injure approximately 1.3 million people annually in the United States alone and cost an estimated $42 billion globally (WHO, 2017). Illegible handwriting is a well-documented contributor: 35.7% of handwritten prescriptions contain errors, compared to just 2.5% of electronic ones (Albarrak et al., 2014).

A study of 4,183 prescriptions found that 10.21% of them are illegible, and 19.39% are barely legible (Albalushi et al., 2023). The global OCR market is projected to reach $32.9 billion by 2030, growing at a 14.8% CAGR (Grand View Research, 2025). Healthcare is one of the fastest-growing verticals driving that growth. But I couldn't find a public benchmark comparing today's open-source OCR engines on handwritten medical text.

So I ran one myself. I tested four engines on 5,578 handwritten prescription word images, and the results surprised me.

Who Are the Four Contenders?

These four engines span three generations of OCR thinking, from traditional pattern matching to specialized deep learning to generative vision-language models.

Tesseract: The Veteran

Google-backed and over 18 years old, Tesseract is the default OCR engine for a generation of developers. It uses an LSTM-based architecture designed primarily for printed text. Stable, well-documented, and runs everywhere, but handwritten cursive is not its strength.

EasyOCR: The Accessible One

Built on a CRNN (Convolutional Recurrent Neural Network) architecture with roughly 10 million parameters, EasyOCR's selling point is simplicity: pip install easyocr and you're recognizing text in 80+ languages. It uses deep learning but remains a traditional detection-recognition pipeline.

PP-OCRv5: The Data-Centric Specialist

Baidu's latest, with just 5 million parameters. PP-OCRv5 uses an SVTR_LCNet architecture with a Guided Training of CTC (GTC) strategy. The real innovation isn't the architecture, though. It's the training data.

The PP-OCRv5 paper (Cui et al., 2026) shows that data quality trumps model scale. They curated 22.6 million training samples by filtering along three dimensions. First, difficulty: they use model confidence as a proxy and found that samples in the [0.95, 0.97] range hit a sweet spot, hard enough to teach the model something new, but not so hard that the labels are unreliable. Second, accuracy: they cross-check predictions against labels to weed out mislabeled samples. Third, diversity: they cluster training images into 1,000 visual groups using CLIP embeddings and ensure each cluster is represented. Together, these filters yielded 2-3x improvements in handwritten recognition from v3 to v5 without changing the model architecture.

GLM-OCR: The Compact Vision-Language Model

From Zhipu AI and Tsinghua University, GLM-OCR takes a fundamentally different approach. It's a 0.9-billion-parameter multimodal model combining a 0.4B CogViT vision encoder with a 0.5B GLM language decoder (Duan et al., 2026). Rather than traditional CTC or attention-based sequence recognition, it generates text autoregressively, like a language model that reads images.

An important note: 0.9B is compact for a vision-language model. For comparison, Qwen3-VL has 235 billion parameters and GPT-4o is even larger. GLM-OCR was designed for efficiency, using Multi-Token Prediction (MTP) to generate approximately 5.2 tokens per decoding step, yielding a roughly 50% throughput improvement over standard autoregressive generation. It's trained through a 4-stage pipeline that includes supervised fine-tuning and GRPO reinforcement learning.

These four engines represent a clear spectrum: traditional (Tesseract), specialized deep learning (EasyOCR, PP-OCRv5), and generative VLM (GLM-OCR).

What Makes the RxHandBD Dataset So Hard?

We use RxHandBD (Shovon et al., Mendeley Data), a dataset of 5,578 cropped word images extracted from handwritten medical prescriptions written by doctors in Bangladesh. Each image contains a single word with a corresponding ground-truth label.

This dataset is hard for four reasons:

Doctor's handwriting. Enough said.
Medical terminology. Drug names like "Amoxicillin" and "Metformin" alongside dosage notations like "5% dns" and "1+0+1."
Mixed language. English medical terms interspersed with Bangla script.
Inconsistent quality. Varying paper backgrounds, pen types, and image capture conditions.

Here are a few samples to give you a sense of the difficulty range:

From top to bottom: a drug name both modern engines nail ("Ronem"), cases where only one engine succeeds ("Vineet" and "Eylox"), and a close call where GLM-OCR gets closest but still misses ("Ambrox").

One important methodological note: these are pre-cropped word-level images. We're isolating the recognition half of the OCR pipeline, not testing detection (locating text regions on a full page). This means our results reflect recognition accuracy only. Real-world performance also depends on how well each engine detects text regions before recognizing them.

How Did We Run the Benchmark?

Metrics

We evaluate using four complementary metrics:

CER (Character Error Rate): Edit distance between predicted and reference strings, normalized by reference length. If the label says "Amoxicillin" (11 characters) and the OCR outputs "Amoxicilin" (1 deletion), the CER is 1/11 = 0.09. Lower is better.
WER (Word Error Rate): Same concept at the word level. Any mistake in a word counts the entire word as wrong. Lower is better.
Exact Match Rate: The strictest metric. Did the OCR output match the ground truth character-for-character after normalization? Higher is better.
Latency: Wall-clock milliseconds per image, measuring practical throughput.

Setup

All engines run on an Apple Silicon Mac (CPU only). Single run. Default configurations for each engine with no fine-tuning or domain-specific adjustments. PP-OCRv5 runs its full detection + recognition pipeline even on pre-cropped images. GLM-OCR processes each image with the prompt "Text Recognition:".

What Do the Numbers Say?

GLM-OCR achieves the lowest character error rate at 0.328, while PP-OCRv5 leads on word-level accuracy with a WER of 0.789. Both modern engines dramatically outperform the older generation, with exact match rates 8-13x higher than Tesseract or EasyOCR.

Engine	Type	Parameters	CER (lower=better)	WER (lower=better)	Exact Match (higher=better)	Latency (ms/img)
Tesseract	LSTM	n/a	0.785	1.043	2.5%	49
EasyOCR	CRNN	~10M	0.695	1.074	2.6%	14
PP-OCRv5	SVTR+CTC	5M	0.477	0.789	21.4%	103
GLM-OCR	VLM	0.9B	0.328	0.801	32.6%	141

Let me walk through what these numbers actually mean.

Finding 1: Tesseract and EasyOCR Barely Function on Handwriting

Both Tesseract and EasyOCR achieve under 3% exact match, meaning they get fewer than 1 in 40 words perfectly right. Their WER exceeds 1.0, which means on average they produce more errors than there are words. For practical purposes, these engines are unusable on handwritten medical text.

This isn't a knock on either project. Tesseract's LSTM architecture was optimized for printed text, and EasyOCR's CRNN similarly excels on clean, well-formatted inputs. Handwritten medical cursive is simply a different problem.

Finding 2: The Modern Engines Are a Generational Leap

PP-OCRv5 and GLM-OCR both break the 20% exact match barrier, a qualitative jump from the sub-3% performance of the older engines. The gap between "old" and "new" (a roughly 10x improvement in exact match) is far larger than the gap between PP-OCRv5 and GLM-OCR themselves.

Finding 3: Why Does GLM-OCR Win on Characters but Lose on Words?

This is the most interesting finding. GLM-OCR achieves a CER of 0.328, which is 31% lower than PP-OCRv5's 0.477. At the character level, the vision-language approach genuinely helps. GLM-OCR can use its language decoder's knowledge of likely character sequences to infer partially visible characters.

But PP-OCRv5 edges ahead on WER: 0.789 vs 0.801. It makes fewer word-level mistakes.

Why the divergence? My hypothesis: GLM-OCR's autoregressive generation occasionally produces subtle extra tokens or formatting variations. The GLM-OCR paper itself acknowledges "minor stochastic variation in formatting behaviors, particularly in line breaks and whitespace handling" (Duan et al., 2026). These small artifacts barely affect CER but can flip a word from "correct" to "incorrect" in WER/exact-match scoring.

The practical takeaway: which engine is "better" depends on your error metric. If you care about getting as close as possible character-by-character (for downstream spell-correction, for example), GLM-OCR wins. If you need clean word-level outputs with minimal postprocessing, PP-OCRv5 has the edge.

Finding 4: What Are the Real Deployment Trade-offs?

Both engines are fast enough for practical use. PP-OCRv5 processes images at 103ms each and GLM-OCR at 141ms. GLM-OCR's Multi-Token Prediction (generating roughly 5.2 tokens per step instead of one) keeps its inference speed competitive despite the larger model.

The bigger difference is model size and memory footprint. PP-OCRv5's 5M parameters take up roughly 20MB on disk, small enough for a Raspberry Pi or embedded device. GLM-OCR's 0.9B parameters need around 1.8GB at FP16 (less with quantization). Both run on CPU without a GPU, as our Apple Silicon benchmark shows, but GLM-OCR consumes significantly more RAM. The GLM-OCR paper notes that the model "enables deployment in both large-scale and resource-constrained edge scenarios" and supports frameworks like Ollama for local inference (Duan et al., 2026). For deploying across thousands of low-spec workstations, PP-OCRv5's small footprint is a clear advantage. For a centralized server or any machine with a few GB of RAM to spare, GLM-OCR is equally practical.

What Do the Predictions Actually Look Like?

Numbers tell one story. Seeing the actual predictions tells another. Here are four representative samples from the benchmark, hand-picked to illustrate each finding above.

GLM-OCR Nails a Complex Drug Name

"Ceftriaxone" (a common antibiotic), 11 characters long. Tesseract reads "Wau" and EasyOCR produces "@@FNRH," both useless. PP-OCRv5 gets close with "CEFRAXONE" (CER=0.18, missing the 'ti'). Only GLM-OCR reads it perfectly. On long drug names, the language decoder's knowledge of likely character sequences gives it a real edge.

When PP-OCRv5 Is Closer and GLM-OCR Hallucinates

"Zolfin" (a proton pump inhibitor). PP-OCRv5 reads "zolfiu" (CER=0.17, just the last character wrong). GLM-OCR outputs "2016'u" (CER=1.0), completely misreading the word as a number. This is the flip side of the VLM approach: when the handwriting doesn't match patterns in the training data, the language model can steer the output in the wrong direction entirely.

GLM-OCR Nearly Perfect

"Pantonix" (a pantoprazole brand). GLM-OCR outputs "Pantomix" (CER=0.125, one character off). PP-OCRv5 gets "Pantomy" (CER=0.375). Both are close, but GLM-OCR is three times more accurate by CER. Neither gets an exact match, which is the pattern behind Finding 3: GLM-OCR consistently gets closer character-by-character, even when neither engine gets the word exactly right.

When the VLM Goes Off the Rails

"Fexo" (fexofenadine, an antihistamine). EasyOCR reads "Texo" (CER=0.25) and PP-OCRv5 reads "-exo" (CER=0.25), both reasonable attempts. GLM-OCR outputs LaTeX math notation: $ \sqrt{e} x_{0} $ (CER=4.75). This is a rare but real failure mode of generative VLMs: the model interprets the handwriting as a math expression instead of text. It produced more characters than the ground truth has, which is why CER exceeds 1.0.

What Are the Limitations?

Single dataset, single run. RxHandBD is one dataset of Bangladeshi prescriptions. US, European, or East Asian handwriting styles may produce different rankings. We don't have confidence intervals from multiple runs.
Word-level only. We tested recognition on pre-cropped word images, not full-page detection + recognition. Real-world performance depends on the complete pipeline.
CPU-only. All engines ran on CPU. GPU acceleration could significantly change the latency picture, particularly for GLM-OCR.
Default configs. No engine was fine-tuned on medical data. Domain-specific adaptation could improve all of them.
No clinical validation. OCR accuracy and clinical safety are different things. A 32.6% exact match rate is impressive for research, but not nearly sufficient for automated prescription processing without human review.

Conclusion

Modern OCR has made a genuine leap on handwritten medical text. PP-OCRv5 (5M parameters, best word-level accuracy) and GLM-OCR (0.9B parameters, best character-level accuracy) both dramatically outperform Tesseract and EasyOCR.

The two champions represent fundamentally different design philosophies: a data-centric specialized pipeline vs. a compact vision-language model. Yet they arrive at remarkably similar performance levels. Both are open-source and practically deployable.

For practitioners building healthcare OCR systems: these two engines deserve serious evaluation. Start with your specific error tolerance, hardware constraints, and whether you need word-level recognition or full-page document understanding.

For researchers: this is a domain with high clinical impact and, as these results show, plenty of room for improvement. Even the best engine here gets only 1 in 3 words exactly right on doctor handwriting. There's real work left to do.

What datasets or engines should I test next? Let me know in the comments.

References

World Health Organization. "WHO launches global effort to halve medication-related errors in 5 years." March 29, 2017. https://www.who.int/news/item/29-03-2017-who-launches-global-effort-to-halve-medication-related-errors-in-5-years
Albarrak AI, Al Rashidi EA, Fatani RK, Al Ageel SI, Mohammed R. "Assessment of legibility and completeness of handwritten and electronic prescriptions." Saudi Pharmaceutical Journal, 2014. https://pmc.ncbi.nlm.nih.gov/articles/PMC4281619/
Albalushi AK, et al. "Assessment of Legibility of Handwritten Prescriptions and Adherence to W.H.O. Prescription Writing Guidelines." J. of Pharmaceutical Research International, 2023. https://pmc.ncbi.nlm.nih.gov/articles/PMC10686667/
Cui C, Zhang Y, Sun T, et al. "PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks." arXiv:2603.24373, March 2026. https://arxiv.org/abs/2603.24373
Duan S, Xue Y, Wang W, et al. "GLM-OCR Technical Report." arXiv:2603.10910, March 2026. https://arxiv.org/abs/2603.10910
Shovon MSH, et al. "RxHandBD: A Handwritten Prescription Recognition Dataset from Bangladesh." Mendeley Data. https://data.mendeley.com/datasets
Grand View Research. "Optical Character Recognition Market Analysis." 2025. https://www.grandviewresearch.com/industry-analysis/optical-character-recognition-market

Benchmark conducted independently. No affiliation with any OCR project. All engines evaluated using default configurations.

About the author: Botao Deng is an ML/AI engineer and researcher who builds and evaluates production models.GitHub

Enhanced Paperless-NGX with Paddle OCR + LLM Pipeline

Mon, 06 Oct 2025 09:15:37 +0000

Building a Private AI Document Pipeline with Paperless, PaddleOCR, and LLMs

So this past week I hacked together a little side project to smarten up my Paperless-ngx setup — you know, that self-hosted document management system that eats PDFs and makes them searchable.

Now, Paperless-ngx is solid, don’t get me wrong. But it uses Tesseract for OCR, and honestly... Tesseract is not optimal for anything that's not clean text.

So this just evolved out of need to improve paperless-ngx's OCR capability and to properly classify documents and extract tags. titles and summaries for documents.

This blog is a walkthrough of what I built — what worked, what didn’t, and how it turned into a pretty neat little pipeline with its own microservices. Hope it gives you some ideas.

What I Was Going For

I wanted a system that:

Uses PaddleOCR instead of Tesseract for better OCR output
Runs a local LLM using Ollama to:
- Suggest a smart document title
- Classify the document into a type (invoice, id, tax, etc.)
Pushes that back to Paperless so the doc is nicely searchable and tagged

Everything stays local/private. No exposure to external LLMs. Just Python, containers, compose to stitch everything together.

How It All Works (Now)

After a few iterations, I ended up with a clean microservice setup with each part doing its job:

paperless-ngx: The main document management system (already amazing)
ollama: Runs a local LLM like Mistral (or phi3 or any configurable in env), no cloud stuff
ocr-service: FastAPI service that runs PaddleOCR
pipeline: Python CLI that connects all the dots — downloads doc from Paperless, sends it to OCR and LLM, then updates Paperless with the smart results

That way, each service does one thing and does it well or can be enhanced in isolation. They all run in Docker, talk to each other over the same network, and make one smooth, local AI document workflow.

This leverages paperless-ngx's extensive features and augments it with better OCR and LLM classification capabilities.

The Flow Looks Like This:

 Paperless stores docs in its DB
⬇️
 I run my pipeline CLI: `docker compose run pipeline 42`
⬇️
 pipeline downloads the doc via Paperless API (by ID)
⬇️
 Sends it to the OCR microservice over HTTP
⬇️
 Gets back clean OCR’d text
⬇️
 Sends text to Ollama (LLM) to generate:
    - title
    - document type
⬇️
 Updates Paperless document via PATCH

Everything Runs in Docker

Here's the final list of containers:

paperless → standard Paperless-ngx
redis → required by Paperless
ollama → runs local LLMs like Mistral
ocr-service → FastAPI + PaddleOCR
pipeline → command-line microservice that ties it all together

Architecture Diagram

                   ┌────────────────────────┐
                   │  Paperless-ngx (UI)  │
                   └────────────┬───────────┘
                                │
                    [User notes Document ID]
                                │
                   ┌────────────▼────────────┐
                   │    Pipeline Service    │
                   │ (LLM & Orchestration)   │
                   └────────────┬────────────┘
                                │
           ┌────────────────────┼────────────────────┐
           │                    │                    │
           ▼                    ▼                    ▼
  ┌────────────────┐   ┌────────────────┐   ┌────────────────────┐
  │ Downloads PDF  │   │ Sends to OCR   │   │ Sends OCR text to  │
  │ via Paperless  │   │ microservice   │   │ Ollama LLM (Mistral)│
  └────────────────┘   └────────────────┘   └────────────────────┘
                                                │
                     ◀────────────┬─────────────┘
                                  ▼
                        Title + Type Prediction
                                  │
                        PATCH back to Paperless
                       (update metadata + text)

What Gave Me Trouble

Paperless consumes files from consume/ automatically and moves it — I had to work around that by working only with doc IDs via API as at this time I was more focused on adding OCR/AI features. This one is high on my TODO list.
PaddleOCR kept re-downloading models — made sure models are cached.

Folder Structure

├── docker-compose.yml
├── __init__.py
├── model-cache
├── ocr_service
│   ├── app
│   │   ├── __init__.py
│   │   ├── main.py
│   │   ├── ocr_config.yaml
│   │   └── ocr_engine.py
│   ├── Dockerfile
│   └── requirements.txt
├── paperless-data
│   ├── consume
│   ├── data
│   │   ├── db.sqlite3
│   │   ├── index
│   │   │   ├── _MAIN_63.toc
│   │   │   ├── MAIN_9v88o8vye3gbqub1.seg
│   │   │   ├── MAIN_ui8jrpftrvauh4n1.seg
│   │   │   ├── MAIN_wceagdbh71brm5wg.seg
│   │   │   └── MAIN_WRITELOCK
│   │   ├── log
│   │   │   └── celery.log.1
│   │   └── migration_lock
│   └── media
│       ├── documents
│       │   ├── archive
│       │   │   ├── 0000009.pdf
│       │   │   └── 0000016.pdf
│       │   ├── originals
│       │   │   ├── 0000009.jpg
│       │   │   └── 0000016.pdf
│       │   └── thumbnails
│       │       ├── 0000009.webp
│       │       └── 0000016.webp
│       └── media.lock
├── pipeline_service
│   ├── app
│   │   ├── api_client.py
│   │   ├── __init__.py
│   │   ├── llm_processor.py
│   │   ├── main.py
│   │   └── watcher.py
│   ├── Dockerfile
│   ├── __init__.py
│   ├── logger.py
│   ├── prompts
│   │   └── classify_title.txt
│   ├── requirements.txt
│   └── test.py
└── README.md

✅ What's Next?

I might:

Run the pipeline automatically when a new doc lands
Add authentication for OCR and pipeline service (utilizing paperless-ngx's token auth?)
Improve performance of OCR service, perhaps using other language (Go, Rust)
Add document summarization via LLM
Extract metadata like amount, date, sender
Hook into Paperless tags and correspondents

Wrapping Up

If you need a document management system:

That ingests all kinds off docs PDFs, images etc.
That reliably extracts text from varied kind of docs.
That classifies, tags and summarizes docs using LLM.
That keeps stuff private - no exposure to external LLMs.
It comes with features loaded from paperless-ngx.

Then this might be an appealing setup.

It’s Python all the way down. No rocket science — just containers, OCR, and a private LLM.

Questions or inputs are welcome.