Computer-Use AI Agent Setup: 13 Steps, 90 Min [2026]

Every major AI lab shipped a computer-use capable model in September 2026. OpenAI put GPT-6 Astra behind a computer-use workflow on September 3, Anthropic pushed Claude Opus 5.5 out on September 22, and Google followed with Gemini 3.8 Flash and Gemini 3.8 Flash Cyber on September 2. The adoption curve backs up the hype: Acronis reported that 9.4% of the Windows PCs it monitors were already running some form of AI-agent tooling between June and September 2026, according to the security vendor’s October 2026 findings, and search interest in “computer use agent” keeps climbing right alongside it, which tells you developers are trying to build these things faster than anyone is writing clear instructions for them. Most of what’s published online either compares benchmark scores across models or demonstrates a polished demo without showing the plumbing that makes it work.

This tutorial fixes that gap. You’ll build a working computer-use agent from scratch, in Python, using OpenAI’s computer-use-preview model through the Responses API, then extend it with a Claude fallback path. By the end you’ll have a CLI tool that takes a plain-English instruction, opens a browser, and completes the task by looking at screenshots and clicking around like a person would. Every code block below is meant to be copied, run, and modified, not just read. You’ll also get a full cost breakdown, a troubleshooting list built from the failure modes that actually trip people up, and a clear answer to a question that trips up a lot of early builds: which model in this new lineup is actually built to click and type, and which one just talks about it.

·

Google · Preferred Sources

Don't miss new tech stories on Google

Add Tech Insider once in the Google app and our stories appear in your news suggestions.

Add Now

What Is a Computer-Use AI Agent?

A computer-use agent is a model that doesn’t just answer questions in text. It looks at a screenshot, decides on a mouse or keyboard action, waits for that action to execute, looks at the new screenshot, and repeats until the task is done. That loop, perception then action then perception again, is the entire mechanism. It’s a meaningfully different design from a chatbot or even a standard tool-calling agent that hits defined API endpoints. A computer-use agent operates the same interface a human does: it clicks buttons, fills text fields, scrolls pages, and reads whatever appears on screen, without needing a custom integration for every website or app.

This matters for automation because most business software doesn’t expose a clean API. Internal dashboards, legacy admin panels, and one-off web forms rarely have documented endpoints. A computer-use agent skips that problem entirely by treating the screen itself as the interface. The tradeoff is speed and cost: every step requires a fresh screenshot and a fresh model call, so these agents are slower and pricier per task than a direct API integration would be. You reach for a computer-use agent when there’s no API to call, not as a default choice over one.

Why Computer-Use Agents Matter Right Now

Bill Gates argued that this shift is bigger than it looks on the surface. “Agents are not only going to change how everyone interacts with computers. They’re also going to upend the software industry, bringing about the biggest revolution in computing since we went from typing commands to tapping on icons,” Gates wrote on his Gates Notes blog. He framed the appeal in practical terms too: “You won’t have to use different apps for different tasks. You’ll simply tell your device, in everyday language, what you want to do.”

The model landscape backs that framing up. OpenAI’s GPT-6 Astra launched as a computer-use flagship rather than a pure chat model, rolling out across ChatGPT Plus, Pro, Business, and Enterprise tiers as well as the API and AWS. Days later, OpenAI followed with lower-cost siblings GPT-6 Sol and GPT-6 Luna on September 22, both built on methods similar to Astra and available through the API, ChatGPT Work, and Codex; OpenAI then refreshed that tier again on September 29 with GPT-6.1 Sol, priced at $2 per million input tokens and $10 per million output tokens, according to MarkTechPost’s October 2026 coverage. Anthropic’s Claude Opus 5.5 arrived the same day, positioned for long-running agentic coding and knowledge work. Google shipped Gemini 3.8 Flash and the security-focused Gemini 3.8 Flash Cyber variant, plus Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking for real-time voice dialogue. DeepSeek added V4.1 Flash on September 10 with sharply lower pricing and what the company describes as substantially enhanced agentic capabilities. Fourteen labs shipped 27 distinct model releases in just the first 18 days of September 2026, according to tracking data compiled across the industry. That pace is exactly why a clear, tested build guide matters more now than a comparison of specs.

Prerequisites: Tools, Accounts, and Exact Versions You Need

Before you write a line of code, get these in place. Nothing here is optional; skipping a step just means you’ll hit an error two steps later and have to backtrack.

Requirement	Version / Detail	Notes
Python	3.10 or newer	Required for the async Playwright API used in this tutorial
OpenAI Python SDK	Latest release from PyPI	Install with pip install --upgrade openai; the Responses API is required, Chat Completions won’t work
Playwright for Python	Latest release	Acts as the “hands” that execute clicks, typing, and scrolling in a real browser
OpenAI API key	Account with billing enabled	Computer-use-preview is billed separately from standard chat models
Anthropic API key (optional)	Account with billing enabled	Needed only for the Claude fallback path in Step 13
Operating system	macOS, Windows, or Linux	Playwright’s bundled Chromium removes most OS-specific quirks
RAM	8GB minimum, 16GB recommended	Headed Chromium plus a Python process is heavier than it looks

You do not need a GPU, a Docker setup, or a cloud VM to follow this tutorial. Everything runs locally against your own browser instance and calls the model APIs over HTTPS.

Steps 1-3: Set Up Your Environment and API Access

Step 1: Create a Clean Virtual Environment

Start in a fresh directory and isolate your dependencies. Computer-use agents pull in several packages that can clash with other projects if you install them globally.

mkdir computer-use-agent
cd computer-use-agent
python3 -m venv venv
source venv/bin/activate
python -m pip install --upgrade pip

On Windows, activate with venv\Scripts\activate instead of the source command above.

Step 2: Install Core Dependencies

Install the OpenAI SDK and Playwright, then download Playwright’s Chromium binary. This last step is easy to forget and is the single most common reason a first run fails.

pip install --upgrade openai playwright python-dotenv
playwright install chromium

The Playwright Python documentation covers additional browser targets if you want Firefox or WebKit instead of Chromium, but Chromium is the safest default for a computer-use agent since it renders the widest range of sites predictably.

Step 3: Store Your API Keys Securely

Create a .env file in your project root. Never hardcode keys directly in your script, and never commit this file to version control.

OPENAI_API_KEY=sk-your-key-here
ANTHROPIC_API_KEY=sk-ant-your-key-here

Add .env to a .gitignore file immediately, before you write any code that reads from it. If you’re using an OpenAI account for the first time with the computer-use-preview model, check that computer-use tools are enabled on your account tier – some organization accounts restrict this tool by default because it can control a live browser session.

Steps 4-6: Build the Perception-Action Loop

Step 4: Launch a Controlled Browser Session

The agent needs something to act on. Rather than controlling your entire desktop with tools like pyautogui, this tutorial uses Playwright to drive a sandboxed browser window. This is safer, easier to debug, and matches how OpenAI’s own computer-use documentation demonstrates the environment: "browser" setting, which is one of the modes the tool supports alongside desktop OS environments.

import asyncio
from playwright.async_api import async_playwright

async def launch_browser():
 playwright = await async_playwright().start()
 browser = await playwright.chromium.launch(headless=False)
 page = await browser.new_page(viewport={"width": 1280, "height": 800})
 await page.goto("https://example.com")
 return playwright, browser, page

Keep headless=False while you’re developing so you can watch what the agent is actually doing. Switch to headless mode only once you trust the agent’s behavior on a given task.

Step 5: Capture and Encode Screenshots

Every turn of the loop starts with a fresh screenshot, encoded as base64 so it can be sent to the model as an image.

import base64

async def capture_screenshot(page):
 screenshot_bytes = await page.screenshot()
 return base64.b64encode(screenshot_bytes).decode("utf-8")

Resolution matters here. Match the display_width and display_height you declare to the model with your actual Playwright viewport. A mismatch is one of the most common sources of misclicks, covered in the troubleshooting section below.

Step 6: Send the First Request to computer-use-preview

With a screenshot in hand, call the OpenAI Responses API. The official computer-use tool documentation is worth bookmarking since the tool’s supported parameters evolve as OpenAI updates the model.

from openai import OpenAI

client = OpenAI()

def get_next_action(task_instruction, screenshot_b64=None, previous_response_id=None):
 tools = [{
 "type": "computer_use_preview",
 "display_width": 1280,
 "display_height": 800,
 "environment": "browser"
 }]

 if previous_response_id is None:
 input_payload = [{"role": "user", "content": task_instruction}]
 else:
 input_payload = [{
 "type": "computer_call_output",
 "call_id": "PLACEHOLDER",
 "output": {
 "type": "input_image",
 "image_url": f"data:image/png;base64,{screenshot_b64}"
 }
 }]

 response = client.responses.create(
 model="computer-use-preview",
 tools=tools,
 input=input_payload,
 previous_response_id=previous_response_id,
 truncation="auto"
 )
 return response

Note the model string: computer-use-preview, not gpt-6-astra. GPT-6 Astra is OpenAI’s general-purpose flagship and is not confirmed to accept the computer-use tool directly. The dedicated computer-use-preview model is the one built and priced specifically for this action loop, currently billed at roughly $3 per million input tokens and $12 per million output tokens according to OpenAI’s published pricing. Keep that distinction in mind before you architect a project around the wrong model name.

Steps 7-9: Parse, Execute, and Loop the Actions

Step 7: Parse the Model’s Response

The response includes one or more computer_call items. Each contains an action object plus a call_id you’ll need to send back with the next screenshot.

def extract_computer_call(response):
 for item in response.output:
 if item.type == "computer_call":
 return item.call_id, item.action
 return None, None

If extract_computer_call returns None, None, the model likely finished the task and returned a plain text summary instead of another action. Check response.output_text in that case before assuming something broke.

Step 8: Execute the Action in the Browser

This is the “hands” of the agent: a function that translates the model’s decision into a real Playwright command.

async def execute_action(page, action):
 action_type = action.get("type")

 if action_type == "click":
 await page.mouse.click(action["x"], action["y"])
 elif action_type == "double_click":
 await page.mouse.dblclick(action["x"], action["y"])
 elif action_type == "type":
 await page.keyboard.type(action["text"])
 elif action_type == "keypress":
 for key in action["keys"]:
 await page.keyboard.press(key)
 elif action_type == "scroll":
 await page.mouse.wheel(action.get("scroll_x", 0), action.get("scroll_y", 0))
 elif action_type == "wait":
 await asyncio.sleep(1.5)
 else:
 print(f"Unhandled action type: {action_type}")

This handles the core actions you’ll see most often: click, double-click, type, keypress, scroll, and wait. Consult OpenAI’s documentation for the complete, current action set since new action types occasionally get added as the tool matures.

Understanding the Full Action Schema

Beyond the six handled in the function above, computer-use models can also return move actions, which reposition the cursor without clicking (useful for triggering hover states like dropdown menus), and drag actions, which take a path of coordinates for click-and-drag interactions such as reordering a list or adjusting a slider. Each action arrives as a plain dictionary, which is why execute_action is written as a simple if/elif chain rather than a class hierarchy. Resist the urge to over-engineer this part early on. A flat dispatch function is easier to debug when you’re staring at a log file trying to figure out why turn six did something unexpected, and it’s trivial to refactor into a lookup table once your action set stabilizes.

One detail worth flagging: the model doesn’t see your DOM, your HTML, or your accessibility tree. It sees pixels. That means visually similar elements, two buttons that look almost identical but sit a few pixels apart, are a genuine source of confusion for the model, not just for your code. Padding your UI targets with generous click areas and avoiding dense, cluttered layouts on any page you expect an agent to operate against will cut down on misclicks more effectively than tuning your prompt.

Step 9: Close the Loop

Now wire steps 6 through 8 together into a running cycle: get an action, execute it, screenshot the result, send it back, repeat.

async def run_agent_loop(page, task_instruction, max_turns=25):
 response = get_next_action(task_instruction)
 turns = 0

 while turns < max_turns:
 call_id, action = extract_computer_call(response)
 if action is None:
 print("Task complete:", response.output_text)
 break

 await execute_action(page, action)
 await page.wait_for_timeout(500)
 screenshot_b64 = await capture_screenshot(page)

 response = get_next_action(
 task_instruction,
 screenshot_b64=screenshot_b64,
 previous_response_id=response.id
 )
 turns += 1

 if turns >= max_turns:
 print("Stopped: max turns reached without completion.")

Simon Willison, who writes extensively about agentic engineering patterns, describes why loops like this are so effective at grinding through multi-step work. “The thing about agentic coding is that agents grind problems into dust,” Willison wrote in a recent conversation on agentic engineering. That framing applies just as well to computer-use loops: the agent doesn’t need to plan the whole task perfectly upfront, it just needs to take one reasonable step at a time and correct course as new screenshots arrive.

The max_turns cap is not optional. Without it, a confused agent can loop indefinitely, burning tokens on every iteration. Twenty-five turns is a reasonable ceiling for most single-page or short-workflow tasks; raise it only for genuinely multi-page processes.

Steps 10-13: Add Guardrails, Test, Log, and Ship

Step 10: Add Human Confirmation for Sensitive Actions

Computer-use models surface pending_safety_checks in the response when an action could have real-world consequences, such as submitting a payment form or deleting data. Never auto-approve these without a review step.

def requires_confirmation(response):
 for item in response.output:
 if item.type == "computer_call" and getattr(item, "pending_safety_checks", None):
 return item.pending_safety_checks
 return None

def ask_human_to_confirm(checks):
 for check in checks:
 print(f"SAFETY CHECK: {check}")
 answer = input("Approve this action? (y/n): ")
 return answer.strip().lower() == "y"

This kind of deliberate checkpoint reflects a broader lesson about where AI agents are genuinely reliable versus where they still need a human backstop. An Anthropic engineer put it plainly when describing how the company’s own staff use agentic tools day to day: “I’m primarily using AI in cases where I know what the answer should be or should look like. I developed that ability by doing software engineering ‘the hard way,'” the engineer said in coverage of Anthropic’s 2026 agentic coding trends report. The same principle applies to computer-use agents: you supervise them well when you already know roughly what correct looks like, which is exactly why the confirmation step matters most on unfamiliar or high-stakes tasks.

Step 11: Test on a Real, Low-Stakes Task

Pick something with a clearly checkable outcome for your first live test, such as filling out a contact form on a site you control or navigating a documentation site to find a specific page. Run it with headless=False so you can watch every click.

async def main():
 playwright, browser, page = await launch_browser()
 try:
 await run_agent_loop(page, "Find the pricing page and report the cheapest plan listed.")
 finally:
 await browser.close()
 await playwright.stop()

asyncio.run(main())

Expect your first few runs to fail in small ways: a misclick on a dropdown, a form field that didn’t register a keypress, or a cookie-consent banner blocking the agent’s view. That’s normal and part of why the max-turns and logging steps below exist.

Step 12: Add Structured Logging

Without logs, debugging a failed run means guessing. Log every action, timestamp, and turn count to a file so you can replay exactly what happened.

import json
import logging

logging.basicConfig(
 filename="agent_run.log",
 level=logging.INFO,
 format="%(asctime)s %(message)s"
)

def log_action(turn, action, call_id):
 logging.info(json.dumps({"turn": turn, "call_id": call_id, "action": action}))

Call log_action right before execute_action in your loop. When something goes wrong three turns in, this log tells you exactly which coordinates or which text string the model chose, rather than leaving you to reconstruct it from memory.

Step 13: Package It as a Reusable CLI Tool

Wrap the whole thing behind argparse so you can run different tasks without editing the script each time.

import argparse

def parse_args():
 parser = argparse.ArgumentParser(description="Run a computer-use AI agent task.")
 parser.add_argument("task", type=str, help="Plain-English task for the agent to complete")
 parser.add_argument("--max-turns", type=int, default=25)
 parser.add_argument("--headless", action="store_true")
 return parser.parse_args()

From here you can run python agent.py "log into the dashboard and export last month's report" --max-turns 40 and iterate on prompts without touching the underlying loop code.

The Complete Working Project

Here’s every piece from Steps 1 through 13 combined into a single runnable file. Save this as agent.py in your project directory alongside the .env file from Step 3.

import asyncio
import base64
import json
import logging
import argparse
from dotenv import load_dotenv
from openai import OpenAI
from playwright.async_api import async_playwright

load_dotenv()
client = OpenAI()

logging.basicConfig(
 filename="agent_run.log",
 level=logging.INFO,
 format="%(asctime)s %(message)s"
)

async def launch_browser(headless=False):
 playwright = await async_playwright().start()
 browser = await playwright.chromium.launch(headless=headless)
 page = await browser.new_page(viewport={"width": 1280, "height": 800})
 await page.goto("https://example.com")
 return playwright, browser, page

async def capture_screenshot(page):
 screenshot_bytes = await page.screenshot()
 return base64.b64encode(screenshot_bytes).decode("utf-8")

def get_next_action(task_instruction, screenshot_b64=None, previous_response_id=None, call_id=None):
 tools = [{
 "type": "computer_use_preview",
 "display_width": 1280,
 "display_height": 800,
 "environment": "browser"
 }]

 if previous_response_id is None:
 input_payload = [{"role": "user", "content": task_instruction}]
 else:
 input_payload = [{
 "type": "computer_call_output",
 "call_id": call_id,
 "output": {
 "type": "input_image",
 "image_url": f"data:image/png;base64,{screenshot_b64}"
 }
 }]

 return client.responses.create(
 model="computer-use-preview",
 tools=tools,
 input=input_payload,
 previous_response_id=previous_response_id,
 truncation="auto"
 )

def extract_computer_call(response):
 for item in response.output:
 if item.type == "computer_call":
 return item.call_id, item.action
 return None, None

async def execute_action(page, action):
 action_type = action.get("type")
 if action_type == "click":
 await page.mouse.click(action["x"], action["y"])
 elif action_type == "double_click":
 await page.mouse.dblclick(action["x"], action["y"])
 elif action_type == "type":
 await page.keyboard.type(action["text"])
 elif action_type == "keypress":
 for key in action["keys"]:
 await page.keyboard.press(key)
 elif action_type == "scroll":
 await page.mouse.wheel(action.get("scroll_x", 0), action.get("scroll_y", 0))
 elif action_type == "wait":
 await asyncio.sleep(1.5)
 else:
 print(f"Unhandled action type: {action_type}")

async def run_agent_loop(page, task_instruction, max_turns=25):
 response = get_next_action(task_instruction)
 turns = 0
 call_id = None

 while turns < max_turns:
 call_id, action = extract_computer_call(response)
 if action is None:
 print("Task complete:", response.output_text)
 break

 logging.info(json.dumps({"turn": turns, "call_id": call_id, "action": action}))
 await execute_action(page, action)
 await page.wait_for_timeout(500)
 screenshot_b64 = await capture_screenshot(page)

 response = get_next_action(
 task_instruction,
 screenshot_b64=screenshot_b64,
 previous_response_id=response.id,
 call_id=call_id
 )
 turns += 1

 if turns >= max_turns:
 print("Stopped: max turns reached without completion.")

async def main():
 parser = argparse.ArgumentParser(description="Run a computer-use AI agent task.")
 parser.add_argument("task", type=str)
 parser.add_argument("--max-turns", type=int, default=25)
 parser.add_argument("--headless", action="store_true")
 args = parser.parse_args()

 playwright, browser, page = await launch_browser(headless=args.headless)
 try:
 await run_agent_loop(page, args.task, max_turns=args.max_turns)
 finally:
 await browser.close()
 await playwright.stop()

if __name__ == "__main__":
 asyncio.run(main())

Run it with python agent.py "search for a Python tutorial and open the first result" and watch the browser window complete the task step by step. Expect output in your terminal similar to a running turn counter followed by a final line reading Task complete: and a short summary from the model.

GPT-6 Astra vs Claude Opus 5.5 vs Gemini 3.8 Flash for Agent Backends

Not every model in this generation plays the same role in a computer-use pipeline. Some are built to drive the screen directly, others are better suited to planning, reasoning, or cheap high-volume orchestration around the agent. On OSWorld 2.0, the benchmark the industry now uses to grade exactly this kind of screen-operating skill, GPT-6 Astra scored 72.6% in September 2026 testing and finished its tasks in about 40 minutes on average, roughly half the 75 minutes GPT-6 Sol needed on the same benchmark, per CodingFleet’s September 2026 numbers. If you’re already comparing raw capability across this model generation, the earlier breakdown of GPT-6 Astra vs Gemini 3.8 vs DeepSeek V4.1 Flash is worth reading alongside this table.

Model	Confirmed Computer-Use Support	Input Price / M Tokens	Output Price / M Tokens	Best Role in an Agent Stack
OpenAI computer-use-preview	Yes – dedicated tool via Responses API	~$3	~$12	Direct screen/browser actuation
GPT-6 Astra	Not confirmed for direct computer-use tool use	$10	$50	High-reasoning task planning, not actuation
Claude Opus 5.5	computer_20251124 tool listed on Bedrock; direct API support unconfirmed	$4	$20	Long-running agentic coding alongside computer use
Gemini 3.8 Flash	Not confirmed	$0.75 (through Dec 31, 2026)	$3.75 (through Dec 31, 2026)	Cheap, high-volume orchestration and planning
DeepSeek V4.1 Flash	Not confirmed	$0.30 peak / $0.15 off-peak	$1.20 peak / $0.60 off-peak	Budget reasoning backend for non-visual sub-tasks

The practical takeaway: don’t assume the flagship chat model in any given lab’s lineup is also the computer-use model. OpenAI splits the two roles into separate model names with separate pricing. Anthropic’s computer-use tool support varies by platform and dated tool version, which is exactly the kind of detail that breaks a project if you don’t check it first. If you’re setting up Claude for a different workflow entirely, the Claude Fable 5.1 API setup guide covers standard API access, and the Gemini 3.8 Flash API setup tutorial walks through Google’s side if you want to swap in Gemini for the planning layer of your own agent.

Real-World Use Cases for Computer-Use Agents

The agent you just built is a general-purpose tool, but a few categories of work explain why teams are adopting this pattern faster than almost any other agentic technique released this year.

• Legacy system automation. Internal tools built a decade ago rarely have a REST API. A computer-use agent can operate the same web interface a human employee uses, without a systems integration project.
• QA and regression testing. Instead of hand-coding brittle Selenium scripts tied to specific CSS selectors, you describe the user flow in plain English and let the agent adapt when minor layout changes would otherwise break a hard-coded test.
• Competitive price monitoring. Sites that actively block scrapers with bot detection are harder to automate through a headless script, but a computer-use agent navigating like a real visitor can check pricing pages that resist conventional scraping.
• Multi-step form submission. Government portals, insurance claim forms, and vendor onboarding forms often span several pages with conditional fields. An agent that reads each screen as it appears handles this branching logic without a hand-built state machine.
• Accessibility testing. Running the same task through an agent that only “sees” the rendered page can surface usability problems, confusing labels, ambiguous buttons, that automated accessibility linters miss because they check markup rather than what a real visitor actually perceives.

None of these use cases require modifying the agent code from this tutorial. What changes is the task instruction you pass in and, for anything touching real accounts or money, how tightly you configure the safety-check confirmation from Step 10.

5 Common Pitfalls When Building Computer-Use Agents

• Confusing the flagship model with the computer-use model. Calling gpt-6-astra with a computer-use tool attached will not behave the way computer-use-preview does. Check the model name in every request before debugging anything else.
• Viewport and display size mismatch. If your Playwright viewport is 1280×800 but you declare display_width: 1920, display_height: 1080 to the model, every click coordinate the model returns will land in the wrong place on your actual screen.
• No cap on turns. An agent that gets stuck in a loop, clicking the same disabled button repeatedly, will keep calling the API until you run out of budget or patience. Always set max_turns.
• Ignoring cookie-consent banners and popups. These overlay the page the agent is trying to read and are one of the most common causes of a stalled task in the first few turns. Consider auto-dismissing known consent banners before starting the loop.
• Auto-approving safety checks to “speed things up.” Skipping the human confirmation step in Step 10 removes the one safeguard that catches an agent about to submit a payment, delete a record, or send an email it shouldn’t.

Troubleshooting: 8 Issues You’ll Hit and How to Fix Them

• “Model does not support tool type computer_use_preview” error. Your account or API key doesn’t have computer-use access enabled. Check your organization’s model access settings on OpenAI’s dashboard before assuming the code is wrong.
• Agent clicks in the wrong location every time. Almost always a display-size mismatch between what you told the model and your actual Playwright viewport. Recheck Step 5 and Step 6.
• Loop never terminates and output_text never appears. The task instruction is likely too vague for the model to know when it’s “done.” Rewrite the instruction with a clear, checkable end state, such as “report the price shown” rather than “explore the site.”
• Playwright throws a “Target closed” error mid-run. The browser window was closed manually or crashed. Wrap your loop in a try/except and check page.is_closed() before each action.
• Claude’s computer_20251124 tool request gets rejected on the direct Anthropic API. This is a known platform gap: the tool has been reported as available for Claude Opus 5.5 on Amazon Bedrock but not consistently on the direct Claude API. Check Anthropic’s current computer-use documentation for your specific platform before assuming it’s a code bug.
• Screenshots return blank or white images. The page likely hadn’t finished rendering. Add a short page.wait_for_load_state("networkidle") call before capturing.
• Text typed into the wrong field. The agent’s previous click likely didn’t actually focus the intended input element. Add a short wait after click actions before sending a type action, and verify focus visually during testing with headless=False.
• Costs climbing faster than expected. Every turn resends a full screenshot as image input, which is more token-expensive than plain text. Reduce viewport resolution where the task allows it, and tighten your max_turns ceiling.

What It Actually Costs to Run a Computer-Use Agent

Cost is the detail most tutorials skip, and it’s the one that determines whether this is viable for your use case. Using OpenAI’s published computer-use-preview rate of roughly $3 per million input tokens and $12 per million output tokens as the base, here’s a rough estimate by task volume. These are directional estimates based on typical screenshot and action-turn sizes; actual cost varies with screen resolution, page complexity, and how many turns a task needs to finish.

Usage Scenario	Approx. Turns per Task	Estimated Cost per Task	Estimated Monthly Cost
Single form-fill test (dev/testing)	5-8	$0.02 – $0.05	Under $5 for occasional testing
Daily scheduled task (1x/day)	10-15	$0.05 – $0.10	~$2 – $3
Light production use (50 tasks/day)	10-15 avg	$0.05 – $0.10 each	~$75 – $150
Heavy production use (500 tasks/day)	10-20 avg	$0.05 – $0.15 each	~$750 – $2,250

Compare that against the cheaper Gemini 3.8 Flash and DeepSeek V4.1 Flash rates in the table above and the economics make sense once you understand the roles: those models are ten times cheaper per token, but neither has confirmed computer-use support, so they’re built for the reasoning and planning work around an agent rather than the click-by-click actuation loop itself.

Advanced Tips: Multi-Model Fallback, Caching, and Safety Hardening

Once the base agent works reliably, a few upgrades make it production-worthy rather than a demo.

Add a Claude fallback path. If OpenAI’s API is rate-limited or returns an error, route the same task to Claude Opus 5.5 using the computer_20251124 tool and the computer-use-2025-11-24 beta header, on whichever platform your Anthropic account supports it. Structure your get_next_action function to accept a provider parameter so you can switch backends without rewriting the loop.

Cache the system instruction and task context. If you’re running the same task template repeatedly with different inputs, cache the static portion of your prompt to cut input token costs. Anthropic’s published rates show cache reads at roughly $0.20 per million tokens versus $4 for fresh input tokens on Opus 5.5, which is a meaningful saving at any real volume, and for sub-tasks that don’t need Opus-level reasoning, Anthropic’s Claude Haiku 5.5, which launched October 7, 2026 at just $0.10 per million input tokens and $0.50 per million output tokens according to DevelopersDigest, is worth routing to instead.

Use Gemini or DeepSeek as a cheap pre-flight planner. Before handing a task to the expensive computer-use loop, have a cheaper model like Gemini 3.8 Flash break the instruction into sub-steps and flag anything ambiguous. This catches vague instructions before they burn turns in the actual actuation loop.

Restrict the browser’s network access. Use Playwright’s route interception to block navigation to domains outside an allowlist relevant to the task. This limits the blast radius if the agent misinterprets an instruction and starts clicking around unrelated pages.

Record video of every run. Playwright supports built-in video capture per browser context with a single configuration option. For any agent touching production data, storing a video alongside the structured log from Step 12 turns a disputed outcome from a guessing game into a five-minute review.

Security and Compliance Considerations for Production Agents

A computer-use agent operates with the same access as whoever is logged into the browser session it controls. That means credentials, session cookies, and any sensitive data visible on screen pass through the model provider as part of every screenshot. Treat this the same way you’d treat any third-party service handling production credentials: minimum-privilege accounts, no admin sessions, and a clear audit trail of every action taken.

This is also where engineering discipline matters more than model choice. An InfoWorld analysis of AI coding agents put the underlying risk plainly: “If you truly want to avoid drowning in ‘AI slop,’ you need experienced engineers who can supervise the observability, testing, and review of all that AI-generated code,” the publication wrote in its coverage of AI coding agents. The same standard applies directly to computer-use agents operating on live systems: logging, the human-confirmation gate from Step 10, and a tight max_turns ceiling aren’t optional extras, they’re the baseline for running this safely outside of a local demo. If you’re layering agent tooling into a broader development workflow, the earlier guide on setting up agent skills in Cursor covers complementary guardrails for coding-focused agents, and the OpenAI Realtime API setup guide is a useful next step if you want to pair this agent with a live voice interface.

For teams building agents that operate inside games or interactive environments rather than web browsers, the pattern is similar but the tooling differs. The guide on building AI NPCs in Unity with an LLM backend covers a comparable perception-action loop adapted for a game engine instead of a browser.

Frequently Asked Questions

What’s the difference between a computer-use agent and a regular AI chatbot?

A chatbot answers questions in text. A computer-use agent looks at screenshots and executes mouse and keyboard actions in a loop until a task is complete, operating the same interface a human would rather than calling a predefined API.

Do I need GPT-6 Astra specifically, or can I use computer-use-preview alone?

You only need computer-use-preview for the actuation loop built in this tutorial. GPT-6 Astra is not confirmed to support the computer-use tool directly and plays a different role, better suited to general reasoning and planning tasks rather than clicking through a screen.

Is Claude Opus 5.5’s computer use tool available on the direct Anthropic API?

Support varies by platform. Anthropic’s documentation lists the computer_20251124 tool for Claude Opus 5.5 on Amazon Bedrock specifically, while direct API support is not consistently confirmed as of this writing. Check Anthropic’s current documentation for your exact platform before building around it.

How much does it cost to run a computer-use agent in production?

At OpenAI’s published computer-use-preview rate of roughly $3 per million input tokens and $12 per million output tokens, a typical 10-15 turn task costs somewhere around $0.05 to $0.10. Fifty tasks a day lands around $75 to $150 a month; scale estimates up from there based on your task volume and average turn count.

Is it safe to let an AI agent control my computer?

It’s safer when scoped to a sandboxed browser session rather than your full desktop, and when paired with the human-confirmation step for safety-flagged actions covered in Step 10. Never run a computer-use agent against a session logged into sensitive accounts without a confirmation gate and a hard cap on turns.

Can I use this with Gemini or DeepSeek instead of OpenAI?

Not for the direct click-and-type actuation loop covered here, since neither Gemini 3.8 Flash nor DeepSeek V4.1 Flash has confirmed computer-use tool support as of September 2026. Both work well as cheaper backends for the planning and reasoning layer around an agent, feeding instructions to the computer-use model that actually executes actions.

What’s the difference between browser-environment and desktop-environment computer use?

Browser environment, the mode used throughout this tutorial, scopes the agent to a single browser window driven by Playwright. Desktop environment gives the model control over an entire operating system session, which carries meaningfully higher risk and is generally reserved for sandboxed virtual machines rather than a developer’s main machine.

Do I need Playwright, or can I use pyautogui directly on my OS?

Playwright is the safer, more debuggable choice for browser-based tasks, since it runs in a sandboxed window separate from your main desktop. Tools like pyautogui operate against your actual screen and mouse, which is riskier and harder to constrain, and is better suited to desktop-environment agents running inside an isolated VM rather than a local development machine.