OCR (Optical Character Recognition) enables agents to read and extract text from images, PDFs, scanned documents, screenshots, and handwritten notes, making visual content searchable and actionable.
Overview
The OCR primitive empowers agents with visual text recognition capabilities, allowing them to process and understand text embedded in images and documents. This transforms unstructured visual content into structured, machine-readable data that agents can analyze, index, and act upon. OCR is essential for:- Document Digitization: Convert scanned documents and PDFs into editable text
- Data Extraction: Extract structured data from invoices, receipts, and forms
- Image Analysis: Read text from screenshots, photos, and diagrams
- Accessibility: Make visual content accessible and searchable
- Automation: Process documents automatically without manual transcription
- Multi-language Support: Extract text in multiple languages and scripts
Image to Text
Extract text from images in any format (PNG, JPG, HEIC, WebP)
PDF Processing
Process multi-page PDFs with text and image content
Handwriting Recognition
Recognize handwritten text with high accuracy
Structured Extraction
Extract tables, forms, and structured data from documents
How OCR Works
When you enable OCR for an agent:- Input: Agent receives image file, URL, or base64-encoded image data
- Detection: OCR engine detects text regions in the image
- Recognition: Advanced ML models recognize characters and words
- Layout Analysis: Preserves document structure, tables, and formatting
- Post-processing: Cleans and formats extracted text
- Output: Returns structured text with confidence scores and coordinates
Powered by Vision Models: Agentbase OCR uses state-of-the-art vision models including Claude 3.5 Sonnet and GPT-4 Vision for superior accuracy.
OCR Capabilities
Text Extraction
Extract plain text from any image:Structured Data Extraction
Extract specific fields from forms and documents:Table Recognition
Extract tables with rows and columns preserved:Code Examples
Basic OCR
Extract from URL
Structured Data Extraction
Multi-page PDF Processing
Table Extraction
Handwriting Recognition
Multi-language OCR
Use Cases
1. Invoice Processing
Automate invoice data extraction and entry:2. Receipt Management
Build expense tracking from receipt photos:3. Identity Verification
Extract and verify ID documents:4. Form Processing
Automate form data extraction:5. Document Digitization
Convert scanned archives to searchable text:6. Business Card Scanner
Extract contact information from business cards:Best Practices
Image Quality
Optimize Image Resolution
Optimize Image Resolution
Preprocessing for Better Results
Preprocessing for Better Results
Handle Large Documents
Handle Large Documents
Data Extraction Accuracy
Define Clear Schemas
Define Clear Schemas
Validate Extracted Data
Validate Extracted Data
Handle Ambiguous Cases
Handle Ambiguous Cases
Performance Optimization
Batch Processing
Batch Processing
Cache Results
Cache Results
Selective Processing
Selective Processing
Integration with Other Primitives
With RAG
Index extracted text for semantic search:With Workflow
Automate document processing pipelines:With Custom Tools
Combine OCR with domain-specific tools:Performance Considerations
Processing Speed
- Single Image: 1-3 seconds per page
- Multi-page PDF: 2-5 seconds per page (with parallelization)
- Handwriting: 3-6 seconds per page (more complex processing)
- Large Documents: Process in batches for optimal performance
Cost Optimization
Accuracy vs Speed Tradeoffs
Troubleshooting
Low Accuracy / Incorrect Text
Low Accuracy / Incorrect Text
Problem: OCR extracts incorrect or garbled textSolutions:
- Verify image quality is at least 300 DPI
- Enable preprocessing options (deskew, contrast enhancement)
- Check image orientation is correct
- Try different OCR models for comparison
- For handwriting, ensure handwriting mode is enabled
Tables Not Extracted Correctly
Tables Not Extracted Correctly
Problem: Table structure is lost or malformedSolutions:
- Enable table extraction explicitly
- Use preserve layout option
- Provide clear instructions about table format
- Consider post-processing to validate table structure
Processing Timeout on Large Files
Processing Timeout on Large Files
Problem: Large PDFs timeout before completionSolutions:
- Process in smaller page ranges
- Use parallel processing
- Increase timeout limits
- Consider breaking into separate jobs
Structured Data Extraction Inconsistent
Structured Data Extraction Inconsistent
Problem: Extracted data doesn’t match expected schemaSolutions:
- Define very specific extraction schema
- Provide examples in prompt
- Add validation instructions
- Use stricter typing and format requirements
Advanced Patterns
Document Classification
Classify documents before extraction:Quality Confidence Scoring
Track extraction confidence:Multi-document Correlation
Extract and correlate data across documents:Related Primitives
RAG
Index extracted text for semantic search
Workflow
Automate document processing pipelines
Custom Tools
Integrate with document management systems
File System
Read and write processed documents
Additional Resources
API Reference
Complete OCR API documentation
Supported Formats
List of supported image and document formats
Best Practices
OCR optimization and accuracy tips