Web browser automation for scraping, testing, and web interaction
Agentbase provides agents with full Chrome browser capabilities for navigating websites, interacting with web applications, extracting data, and automating web workflows.
The Browser primitive gives agents access to a full Google Chrome browser running within their execution environment. This enables sophisticated web automation, data extraction, and testing capabilities without any additional setup:
Web Navigation: Visit any website and navigate through pages
Element Interaction: Click buttons, fill forms, submit data
Data Extraction: Scrape content, tables, images, and structured data
JavaScript Execution: Run custom JavaScript in the browser context
Screenshots: Capture full pages or specific elements
Session Management: Handle cookies, authentication, and multi-page workflows
Full Chrome Browser
Google Chrome 140.0.7339.127 with complete rendering and JavaScript support
Headless & GUI Modes
Run browser invisibly for automation or with display for debugging
Automation Tools
Built-in support for Selenium, Playwright, and Puppeteer
Internet Access
Full access to the web with cookies, sessions, and authentication
Agents use browser capabilities automatically when you request web-based tasks:
Copy
// Agent automatically uses browser toolsconst result = await agentbase.runAgent({ message: "Navigate to example.com and extract the main heading"});// Browser is launched, page is loaded, data is extracted
// Simple page visitconst visit = await agentbase.runAgent({ message: "Navigate to https://example.com and show the page title"});// Navigate and extractconst extract = await agentbase.runAgent({ message: "Go to news.ycombinator.com and extract the top 5 story titles"});// Follow linksconst follow = await agentbase.runAgent({ message: "Visit the homepage, click on the About link, and show the content"});
// Scrape structured dataconst scrape = await agentbase.runAgent({ message: `Visit https://example-store.com/products and extract: - Product names - Prices - Availability status Save the data as products.json`});// Multi-page scrapingconst multiPage = await agentbase.runAgent({ message: `Scrape the first 3 pages of search results: 1. Visit the search page 2. Extract results from page 1 3. Click next and scrape page 2 4. Click next and scrape page 3 5. Combine all results into a CSV file`});// Table extractionconst table = await agentbase.runAgent({ message: "Extract the data table from the page and convert to CSV"});
// Fill and submit formconst form = await agentbase.runAgent({ message: `Fill out the contact form: - Name: John Doe - Email: [email protected] - Message: Hello, I have a question Then submit the form and confirm submission`});// Login workflowconst login = await agentbase.runAgent({ message: `Login to the website: 1. Navigate to /login 2. Enter username: testuser 3. Enter password: testpass 4. Click submit 5. Verify successful login`});// Search and filterconst search = await agentbase.runAgent({ message: `Use the search and filter: 1. Enter "laptops" in search box 2. Select "Price: Low to High" from dropdown 3. Check "In Stock Only" checkbox 4. Click Search 5. Extract the results`});
// Full page screenshotconst screenshot = await agentbase.runAgent({ message: "Navigate to homepage and take a full page screenshot"});// Element screenshotconst element = await agentbase.runAgent({ message: "Take a screenshot of just the navigation menu"});// Multiple screenshotsconst comparison = await agentbase.runAgent({ message: `Take screenshots for visual testing: 1. Desktop view (1920x1080) 2. Tablet view (768x1024) 3. Mobile view (375x667) Save all three screenshots`});// Before/after comparisonconst changes = await agentbase.runAgent({ message: `Compare page changes: 1. Take screenshot of current state 2. Click "Show More" button 3. Take screenshot of new state 4. Highlight differences`});
// Execute custom JavaScriptconst executeJS = await agentbase.runAgent({ message: "Run JavaScript to get the page's scroll height"});// Interact with page JavaScriptconst interact = await agentbase.runAgent({ message: "Execute JavaScript to trigger the page's custom modal"});// Extract dynamic dataconst dynamic = await agentbase.runAgent({ message: `Use JavaScript to: 1. Wait for data to load via AJAX 2. Extract the dynamically loaded content 3. Return as JSON`});// Modify page contentconst modify = await agentbase.runAgent({ message: "Execute JavaScript to highlight all external links on the page"});
const competitive = await agentbase.runAgent({ message: `Competitor analysis: 1. Visit competitor website 2. Extract current pricing for all products 3. Screenshot their homepage 4. Extract featured products 5. Save all data to competitor_data.json 6. Compare with our pricing`});
const testing = await agentbase.runAgent({ message: `Test user registration flow: 1. Navigate to /signup 2. Fill registration form with test data 3. Submit form 4. Verify email confirmation page 5. Take screenshot of confirmation 6. Check for any errors or issues 7. Generate test report`});
const collection = await agentbase.runAgent({ message: `Collect real estate listings: 1. Visit property listing site 2. Search for "apartments in San Francisco" 3. Extract all listings (address, price, bedrooms, etc.) 4. Visit each listing for detailed info 5. Save all data to properties.csv 6. Create summary statistics`});
const automation = await agentbase.runAgent({ message: `Submit bulk job applications: 1. Read applicant_data.csv 2. For each application: - Navigate to application form - Fill in all fields from CSV - Upload resume.pdf - Submit form - Save confirmation number 3. Generate submission report`});
// Good: Wait for dynamic contentconst good = await agentbase.runAgent({ message: `Visit the page and wait for the products to load before extracting data`});// Avoid: Extracting before content loadsconst bad = await agentbase.runAgent({ message: "Quickly visit page and extract data"});
Handle Pagination Correctly
Copy
// Good: Navigate through all pagesconst good = await agentbase.runAgent({ message: `Scrape all pages: - Extract data from current page - Click next button - Repeat until no more pages - Combine all results`});
Be Specific About Selectors
Copy
// Good: Specific instructionsconst good = await agentbase.runAgent({ message: "Extract prices from elements with class 'product-price'"});// Avoid: Vague instructionsconst bad = await agentbase.runAgent({ message: "Get all the prices"});
Graceful Failures: Instruct agents to handle common web issues like timeouts, missing elements, and navigation errors.
Copy
// Robust error handlingconst robust = await agentbase.runAgent({ message: `Scrape data with error handling: - If page doesn't load, retry up to 3 times - If element not found, log and continue - If data format unexpected, note in output - Generate report of successful and failed extractions`});
Extract all needed data in one visit when possible
Use Headless Mode
Headless is faster - use GUI only for debugging
Parallel Scraping
Scrape multiple pages concurrently when appropriate
Cache Responses
Save scraped data to avoid re-scraping
Copy
// Efficient: Extract all data at onceconst efficient = await agentbase.runAgent({ message: `Visit the page and extract: - All product names - All prices - All descriptions - All images In a single visit`});// Inefficient: Multiple visitsconst inefficient = await agentbase.runAgent({ message: "Visit page, get names, then visit again for prices..."});
const combined = await agentbase.runAgent({ message: `Scrape product data and save: 1. Extract data from website 2. Save as products.json 3. Also create products.csv 4. Generate summary report as report.md`});
const automation = await agentbase.runAgent({ message: `Create web scraping script: 1. Install selenium and beautifulsoup4 2. Write Python script for scraping 3. Run the script 4. Save results to database`});
const research = await agentbase.runAgent({ message: `Research topic: 1. Search web for "topic" 2. Visit top 5 results 3. Extract key information from each 4. Create comprehensive summary`});
const session = await agentbase.runAgent({ message: `Manage sessions: 1. Login and save cookies 2. Navigate to protected pages using saved session 3. Extract user-specific data 4. Logout when done`});
const popups = await agentbase.runAgent({ message: `Handle popups: 1. Navigate to page 2. If popup appears, close it 3. If alert shows, accept it 4. Continue with main task`});
const mobile = await agentbase.runAgent({ message: `Test mobile view: 1. Set viewport to mobile size (375x667) 2. Set mobile user agent 3. Navigate to website 4. Take screenshots 5. Extract mobile-specific content`});
const fix = await agentbase.runAgent({ message: `If page doesn't load: - Wait up to 30 seconds - Check for error messages - Try loading a different page to test connection`});
Element Not Found
Problem: Cannot find element to interact withSolutions:
Wait for element to appear
Check selector is correct
Verify element is visible
Check if it’s in an iframe
Copy
const fix = await agentbase.runAgent({ message: `Find element with retry: - Wait for element to appear (up to 10s) - If not found, check page source - List available elements for debugging`});
const fix = await agentbase.runAgent({ message: "Check console for JavaScript errors and try alternative approach"});
Timeout Errors
Problem: Operations timing outSolutions:
Increase wait time
Check network connection
Simplify task
Copy
const fix = await agentbase.runAgent({ message: "If timeout occurs, wait longer and retry with simpler approach"});
Anti-Scraping Measures
Problem: Website blocking automationSolutions:
Add delays between requests
Use realistic user agent
Respect rate limits
Consider alternative data sources
Copy
const respectful = await agentbase.runAgent({ message: `Scrape respectfully: - Add 2-3 second delays between requests - Use standard browser user agent - Limit to reasonable number of pages`});
Remember: Browser capabilities are available automatically when you describe web-based tasks. Agents handle browser management, navigation, and data extraction for you.