Computer Use & Browser Automation

Computer Use gives your agent the ability to browse websites, interact with web applications, extract data, fill forms, and take screenshots — just like a human would.

---

What your agent can do

• Navigate to any URL and interact with the page
• Click buttons, fill forms, scroll, hover
• Extract structured data from web pages
• Take screenshots and describe what’s on screen
• Log into web apps (using credentials from your Secrets Vault)
• Download and process files
• Run terminal commands in an isolated sandbox

---

How it works

Each Computer Use session runs in a fully isolated, ephemeral browser environment:

1. Agent triggers a new browser session
2. Browser navigates to the target URL
3. A screenshot and the page structure are sent back to the agent
4. Agent decides what action to take next (click, type, scroll, extract)
5. Action is executed and a new screenshot is captured
6. Steps 3–5 repeat until the task is complete
7. Session is automatically closed and discarded

Note

The sandbox is completely isolated — it cannot access your local machine, your files, or any system outside the session. Each session starts fresh.

---

Enabling Computer Use

1. Open your agent in the Console
2. Go to the Settings tab
3. Toggle on Computer Use
4. Save your changes

---

Use cases

Web Data Extraction

Task: Go to https://news.ycombinator.com/front and extract the top 10 stories.
For each story, get:
- Title
- URL
- Number of points
- Number of comments
Return the results as a markdown table.

Form Automation

Task: Go to https://forms.example.com/onboarding and fill in the form:
- Company name: Acme Corp
- Contact: John Smith
- Email: john@acme.com
Submit the form and confirm it was successful.

UI Testing

Task: Test the login flow on https://app.example.com:
1. Navigate to the login page
2. Enter credentials (from secrets vault)
3. Verify you reach the dashboard
4. Take a screenshot of the dashboard
5. Report any errors encountered

Monitoring

Task: Take a screenshot of our status page at https://status.example.com.
If any service shows a non-green status, send an alert to #ops-alerts on Slack
with the screenshot attached.

---

Tips for reliable automation

Be explicit about UI elements

Instead of “click the login button”, say “click the button that says ‘Sign In’ near the top right of the page.”

Add error handling

“If the page takes more than 10 seconds to load, try refreshing once. If it still fails, report back with the URL and a screenshot.”

Test in the panel first

Always test computer use tasks in the interactive Test panel before deploying to a Flow.

Use Secrets Vault for credentials

Store passwords in Account → Secrets and reference them as {{SECRET_NAME}} — never hardcode credentials in instructions.

---

Compute usage

Computer Use sessions are billed against your compute minutes allowance (separate from credits). Each session has a maximum duration before it automatically terminates.

Check your current compute usage in Account → Usage → Compute. See Credits & Billing → for plan limits.