Skip to main content
OCR (Optical Character Recognition) enables agents to read and extract text from images, PDFs, scanned documents, screenshots, and handwritten notes, making visual content searchable and actionable.

Overview

The OCR primitive empowers agents with visual text recognition capabilities, allowing them to process and understand text embedded in images and documents. This transforms unstructured visual content into structured, machine-readable data that agents can analyze, index, and act upon. OCR is essential for:
  • Document Digitization: Convert scanned documents and PDFs into editable text
  • Data Extraction: Extract structured data from invoices, receipts, and forms
  • Image Analysis: Read text from screenshots, photos, and diagrams
  • Accessibility: Make visual content accessible and searchable
  • Automation: Process documents automatically without manual transcription
  • Multi-language Support: Extract text in multiple languages and scripts

Image to Text

Extract text from images in any format (PNG, JPG, HEIC, WebP)

PDF Processing

Process multi-page PDFs with text and image content

Handwriting Recognition

Recognize handwritten text with high accuracy

Structured Extraction

Extract tables, forms, and structured data from documents

How OCR Works

When you enable OCR for an agent:
  1. Input: Agent receives image file, URL, or base64-encoded image data
  2. Detection: OCR engine detects text regions in the image
  3. Recognition: Advanced ML models recognize characters and words
  4. Layout Analysis: Preserves document structure, tables, and formatting
  5. Post-processing: Cleans and formats extracted text
  6. Output: Returns structured text with confidence scores and coordinates
Powered by Vision Models: Agentbase OCR uses state-of-the-art vision models including Claude 3.5 Sonnet and GPT-4 Vision for superior accuracy.

OCR Capabilities

Text Extraction

Extract plain text from any image:

Structured Data Extraction

Extract specific fields from forms and documents:

Table Recognition

Extract tables with rows and columns preserved:

Code Examples

Basic OCR

Extract from URL

Structured Data Extraction

Multi-page PDF Processing

Table Extraction

Handwriting Recognition

Multi-language OCR

Use Cases

1. Invoice Processing

Automate invoice data extraction and entry:

2. Receipt Management

Build expense tracking from receipt photos:

3. Identity Verification

Extract and verify ID documents:

4. Form Processing

Automate form data extraction:

5. Document Digitization

Convert scanned archives to searchable text:

6. Business Card Scanner

Extract contact information from business cards:

Best Practices

Image Quality

Data Extraction Accuracy

Use Schemas for Structured Data: Define extraction schemas to get consistently formatted output and improve accuracy.

Performance Optimization

Integration with Other Primitives

With RAG

Index extracted text for semantic search:
Learn more: RAG Primitive

With Workflow

Automate document processing pipelines:
Learn more: Workflow Primitive

With Custom Tools

Combine OCR with domain-specific tools:
Learn more: Custom Tools Primitive

Performance Considerations

Processing Speed

  • Single Image: 1-3 seconds per page
  • Multi-page PDF: 2-5 seconds per page (with parallelization)
  • Handwriting: 3-6 seconds per page (more complex processing)
  • Large Documents: Process in batches for optimal performance

Cost Optimization

Vision Model Costs: OCR uses vision models which have different pricing than text-only models. See pricing page for details.

Accuracy vs Speed Tradeoffs

Troubleshooting

Problem: OCR extracts incorrect or garbled textSolutions:
  • Verify image quality is at least 300 DPI
  • Enable preprocessing options (deskew, contrast enhancement)
  • Check image orientation is correct
  • Try different OCR models for comparison
  • For handwriting, ensure handwriting mode is enabled
Problem: Table structure is lost or malformedSolutions:
  • Enable table extraction explicitly
  • Use preserve layout option
  • Provide clear instructions about table format
  • Consider post-processing to validate table structure
Problem: Large PDFs timeout before completionSolutions:
  • Process in smaller page ranges
  • Use parallel processing
  • Increase timeout limits
  • Consider breaking into separate jobs
Problem: Extracted data doesn’t match expected schemaSolutions:
  • Define very specific extraction schema
  • Provide examples in prompt
  • Add validation instructions
  • Use stricter typing and format requirements

Advanced Patterns

Document Classification

Classify documents before extraction:

Quality Confidence Scoring

Track extraction confidence:

Multi-document Correlation

Extract and correlate data across documents:

RAG

Index extracted text for semantic search

Workflow

Automate document processing pipelines

Custom Tools

Integrate with document management systems

File System

Read and write processed documents

Additional Resources

API Reference

Complete OCR API documentation

Supported Formats

List of supported image and document formats

Best Practices

OCR optimization and accuracy tips
Pro Tip: For best results, combine OCR with clear extraction instructions and validation logic. The agent can verify extracted data and flag inconsistencies automatically.