Data Connectors enable agents to read from and write to databases, data warehouses, and various data sources, making it easy to work with structured and unstructured data across your organization.
Overview
The Data Connectors primitive provides agents with direct access to your data infrastructure. Whether you’re working with SQL databases, NoSQL stores, data warehouses, or cloud storage, agents can query, analyze, and manipulate data using natural language. Data Connectors are essential for:- Database Access: Query and update SQL and NoSQL databases
- Data Analysis: Analyze data across multiple sources
- Data Synchronization: Keep data in sync across systems
- Reporting: Generate reports from live data
- ETL Operations: Extract, transform, and load data
- Data Validation: Verify data integrity and quality
SQL Databases
Connect to PostgreSQL, MySQL, SQL Server, and more
NoSQL Stores
Access MongoDB, Redis, DynamoDB, and other NoSQL databases
Data Warehouses
Query Snowflake, BigQuery, Redshift, and analytics platforms
Cloud Storage
Read and write to S3, GCS, Azure Blob, and object storage
How Data Connectors Work
When you configure data connectors for an agent:- Connection: Agent establishes secure connection to data source
- Schema Discovery: Automatically detects tables, columns, and relationships
- Query Generation: Converts natural language to appropriate queries (SQL, NoSQL, etc.)
- Execution: Runs queries with proper security and access controls
- Result Processing: Formats and returns results in structured format
- Connection Pooling: Maintains efficient connection management
Security First: All database credentials are encrypted at rest and in transit. Connections use SSL/TLS when available.
Supported Data Sources
SQL Databases
NoSQL Databases
Data Warehouses
Cloud Storage
Code Examples
Basic SQL Query
Connection String
NoSQL Query
Data Warehouse Query
Multiple Data Sources
Write Operations
Cloud Storage Access
Caching Queries
Use Cases
1. Analytics Dashboard
Generate real-time analytics from data warehouse:2. Data Quality Validation
Validate data integrity across systems:3. ETL Pipeline
Extract, transform, and load data:4. Customer Lookup
Build customer service tools:5. Report Generation
Generate custom reports from data:6. Database Migration
Migrate data between databases:Best Practices
Security
Use Read-Only Accounts
Use Read-Only Accounts
Limit Access Scope
Limit Access Scope
Use Connection Pooling
Use Connection Pooling
Enable Query Logging
Enable Query Logging
Performance
Use Query Timeouts
Use Query Timeouts
Limit Result Sets
Limit Result Sets
Use Indexes Effectively
Use Indexes Effectively
Batch Operations
Batch Operations
Data Integrity
Use Transactions
Use Transactions
Validate Before Write
Validate Before Write
Handle Errors Gracefully
Handle Errors Gracefully
Integration with Other Primitives
With RAG
Combine database queries with semantic search:With Workflow
Automate data processing workflows:With Memory
Remember query patterns and preferences:Performance Considerations
Query Optimization
- Query Planning: Analyze query plans before execution
- Index Usage: Ensure queries use appropriate indexes
- Result Limiting: Always limit result sets to needed rows
- Caching: Cache frequently accessed, slowly changing data
Connection Management
- Pool Size: Configure appropriate connection pool sizes
- Connection Reuse: Reuse connections across queries
- Timeout Management: Set appropriate timeouts
- Cleanup: Close idle connections
Cost Optimization
Troubleshooting
Connection Failures
Connection Failures
Problem: Cannot connect to databaseSolutions:
- Verify connection credentials are correct
- Check network connectivity and firewall rules
- Ensure database server is running
- Verify SSL/TLS settings match requirements
- Check IP whitelist if applicable
Query Timeouts
Query Timeouts
Problem: Queries timing outSolutions:
- Increase timeout limits
- Optimize slow queries with indexes
- Reduce result set size
- Use query caching
- Consider breaking into smaller queries
Permission Errors
Permission Errors
Problem: Access denied errorsSolutions:
- Verify user has required permissions
- Check table/schema access rights
- Enable write permissions if needed
- Review database user grants
- Check row-level security policies
Schema Discovery Issues
Schema Discovery Issues
Problem: Agent can’t see tables or columnsSolutions:
- Verify user has permissions to read schema
- Check search_path for PostgreSQL
- Specify schemas explicitly
- Refresh schema cache
Advanced Patterns
Query Result Streaming
Stream large result sets:Multi-tenancy
Isolate data by tenant:Change Data Capture
Monitor database changes:Related Primitives
RAG
Combine with semantic search on database content
Integrations
Connect to external APIs and services
Workflow
Automate data processing workflows
Tasks
Schedule recurring data operations
Additional Resources
API Reference
Complete data connectors API documentation
Supported Databases
Full list of supported data sources
Security Guide
Best practices for secure database access