Project Overview

Learn to use OpenClaw's browser automation capabilities for web data collection, form filling, and automated web interactions.

Browser Tool Basics

// Available browser tools
const browserTools = {
  navigate: 'Go to a URL',
  click: 'Click an element',
  type: 'Type text into inputs',
  screenshot: 'Capture page image',
  extract: 'Get text/attributes from elements',
  wait: 'Wait for element/condition'
};

Step 1: Basic Web Scraping

// "Get today's top stories from Hacker News"

await browser.navigate('https://news.ycombinator.com');

const stories = await browser.extract({
  selector: '.titleline > a',
  limit: 10,
  attributes: ['href', 'innerText']
});

// Format response
"📰 Top 10 Hacker News Stories:
1. [First story title]
2. [Second story title]
..."

Step 2: Dynamic Content Handling

// Wait for dynamic content
await browser.navigate(url);

await browser.wait({
  selector: '.content-loaded',
  timeout: 10000
});

// Now extract content
const data = await browser.extract({
  selector: '.dynamic-content'
});

Step 3: Form Automation

// "Fill out the contact form on example.com"

await browser.navigate('https://example.com/contact');

await browser.type({
  selector: '#name',
  text: 'John Doe'
});

await browser.type({
  selector: '#email', 
  text: '[email protected]'
});

await browser.type({
  selector: '#message',
  text: 'Hello, I have a question...'
});

// Wait for confirmation before submit
"Form filled. Ready to submit?"

Step 4: Screenshot Documentation

// "Take a screenshot of the dashboard"

await browser.navigate(dashboardUrl);

// Wait for charts to load
await browser.wait({ selector: '.charts-loaded' });

const screenshot = await browser.screenshot({
  fullPage: false,
  path: './screenshots/dashboard.png'
});

"📸 Screenshot saved: dashboard.png"

Step 5: Multi-page Navigation

// "Check prices across multiple stores"

const stores = ['store1.com', 'store2.com', 'store3.com'];
const prices = [];

for (const store of stores) {
  await browser.navigate(`https://${store}/product/123`);
  
  const price = await browser.extract({
    selector: '.price',
    attribute: 'innerText'
  });
  
  prices.push({ store, price });
}

// Compare results
"💰 Price Comparison:
Store 1: $29.99
Store 2: $34.99 
Store 3: $27.99 ✨ Best price"

Best Practices

Respect robots.txt and rate limits
Use appropriate delays between requests
Handle errors and timeouts gracefully
Don't scrape sensitive/private data

Conclusion

Browser automation extends OpenClaw's capabilities to interact with any web-based service, enabling powerful automation workflows.