Skip to main content
Data Connectors enable agents to read from and write to databases, data warehouses, and various data sources, making it easy to work with structured and unstructured data across your organization.

Overview

The Data Connectors primitive provides agents with direct access to your data infrastructure. Whether you’re working with SQL databases, NoSQL stores, data warehouses, or cloud storage, agents can query, analyze, and manipulate data using natural language. Data Connectors are essential for:
  • Database Access: Query and update SQL and NoSQL databases
  • Data Analysis: Analyze data across multiple sources
  • Data Synchronization: Keep data in sync across systems
  • Reporting: Generate reports from live data
  • ETL Operations: Extract, transform, and load data
  • Data Validation: Verify data integrity and quality

SQL Databases

Connect to PostgreSQL, MySQL, SQL Server, and more

NoSQL Stores

Access MongoDB, Redis, DynamoDB, and other NoSQL databases

Data Warehouses

Query Snowflake, BigQuery, Redshift, and analytics platforms

Cloud Storage

Read and write to S3, GCS, Azure Blob, and object storage

How Data Connectors Work

When you configure data connectors for an agent:
  1. Connection: Agent establishes secure connection to data source
  2. Schema Discovery: Automatically detects tables, columns, and relationships
  3. Query Generation: Converts natural language to appropriate queries (SQL, NoSQL, etc.)
  4. Execution: Runs queries with proper security and access controls
  5. Result Processing: Formats and returns results in structured format
  6. Connection Pooling: Maintains efficient connection management
Security First: All database credentials are encrypted at rest and in transit. Connections use SSL/TLS when available.

Supported Data Sources

SQL Databases

NoSQL Databases

Data Warehouses

Cloud Storage

Code Examples

Basic SQL Query

Connection String

NoSQL Query

Data Warehouse Query

Multiple Data Sources

Write Operations

Cloud Storage Access

Caching Queries

Use Cases

1. Analytics Dashboard

Generate real-time analytics from data warehouse:

2. Data Quality Validation

Validate data integrity across systems:

3. ETL Pipeline

Extract, transform, and load data:

4. Customer Lookup

Build customer service tools:

5. Report Generation

Generate custom reports from data:

6. Database Migration

Migrate data between databases:

Best Practices

Security

Read-Only by Default: Data connectors are read-only by default. Explicitly enable write permissions only when needed.

Performance

Cache Frequently Accessed Data: Enable caching for queries that run frequently and don’t need real-time data.

Data Integrity

Integration with Other Primitives

With RAG

Combine database queries with semantic search:
Learn more: RAG Primitive

With Workflow

Automate data processing workflows:
Learn more: Workflow Primitive

With Memory

Remember query patterns and preferences:
Learn more: Memory Primitive

Performance Considerations

Query Optimization

  • Query Planning: Analyze query plans before execution
  • Index Usage: Ensure queries use appropriate indexes
  • Result Limiting: Always limit result sets to needed rows
  • Caching: Cache frequently accessed, slowly changing data

Connection Management

  • Pool Size: Configure appropriate connection pool sizes
  • Connection Reuse: Reuse connections across queries
  • Timeout Management: Set appropriate timeouts
  • Cleanup: Close idle connections

Cost Optimization

Monitor Warehouse Usage: Data warehouse queries can be expensive. Monitor usage and optimize expensive queries.

Troubleshooting

Problem: Cannot connect to databaseSolutions:
  • Verify connection credentials are correct
  • Check network connectivity and firewall rules
  • Ensure database server is running
  • Verify SSL/TLS settings match requirements
  • Check IP whitelist if applicable
Problem: Queries timing outSolutions:
  • Increase timeout limits
  • Optimize slow queries with indexes
  • Reduce result set size
  • Use query caching
  • Consider breaking into smaller queries
Problem: Access denied errorsSolutions:
  • Verify user has required permissions
  • Check table/schema access rights
  • Enable write permissions if needed
  • Review database user grants
  • Check row-level security policies
Problem: Agent can’t see tables or columnsSolutions:
  • Verify user has permissions to read schema
  • Check search_path for PostgreSQL
  • Specify schemas explicitly
  • Refresh schema cache

Advanced Patterns

Query Result Streaming

Stream large result sets:

Multi-tenancy

Isolate data by tenant:

Change Data Capture

Monitor database changes:

RAG

Combine with semantic search on database content

Integrations

Connect to external APIs and services

Workflow

Automate data processing workflows

Tasks

Schedule recurring data operations

Additional Resources

API Reference

Complete data connectors API documentation

Supported Databases

Full list of supported data sources

Security Guide

Best practices for secure database access
Pro Tip: Use read-only replicas for analytics queries to avoid impacting production database performance. Agent can automatically route queries to appropriate databases.