Get started with the browser automation tool in hosted agents

Prerequisites

  • An Azure account with an active subscription. If you don't have an Azure subscription, create a free account before you begin.
  • Your Azure account needs the Owner or Contributor roles on a resource group.
  • Azure Developer CLI. If you don't have Azure Developer CLI, see Install the Azure Developer CLI.
    • The azd ai agent extension installed (azd extension install azure.ai.agents). If you don't have the extension installed, when you initialize the starter template or run azd ai agent the extension is installed automatically.

Step 1: Create a Playwright Workspace

To use the browser automation tool, you first need to create a Playwright workspace. (If you have an existing Playwright Workspace, you can skip this step.)

  1. Sign in to the Azure portal by using the credentials for your Azure subscription.
  2. From the portal Home page, search for and select Playwright Workspaces.
  3. Select Create and enter the following information:
    • Subscription
    • Resource group
    • Name
    • Location
  4. Select Review + Create and then select Create. It takes a few minutes to create the workspace. Wait for the portal page to display Your deployment is complete before moving on.

Step 2: Deploy the sample hosted agent

To get started with the browser automation tool in hosted agents, choose one of these samples:

You can use any one of these depending on which framework you want to use to build your hosted agent.

Step 2.1: Initialize the sample hosted agent application

  1. Open a new directory and initialize the agent sample from the sample's azure.yaml by running one of these commands:

    • Microsoft Agent Framework:
    azd ai agent init -t "https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/azure.yaml" --deploy-mode container
    • Bring-your-own Framework:
    azd ai agent init -t "https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/bring-your-own/responses/browser-automation/azure.yaml" --deploy-mode container

  2. The interactive flow prompts for the following details:

    • App directory: The local path to initialize the application.
    • Agent name: Customize the name or accept the default, browser-automation-agent-sample-foundry. The name should be within 1-63 characters, start and end with a letter or number, and contain only letters, numbers, and internal hyphens.
    • Foundry project: Select an existing Foundry project or create a new project. If you use an existing Foundry project, you must have the Foundry Project Manager role at project scope or another role that grants both data-plane and role-assignment permissions. For the full role matrix, see Hosted agent permissions reference.
    • Model: Select the default, gpt-5.4-mini, or another model you can access.
    • Model version: Select the default option.
    • Model SKU: Select an option with available quota that isn't Batch, usually Standard or GlobalStandard.
    • Deployment capacity: Select the default, 10.
    • Deployment name: Select the default, gpt-5.4-mini.

After the application is initialized successfully, you see the following message:

AI agent definition added to your azd project successfully!

Step 2.2: Provision the Azure resources

  1. Set the environment variables to point to your Playwright Workspace

    • Playwright Workspace resource ID: To get the resource ID, open the Playwright Workspace resource, go to the Overview page, select JSON View, and copy the Resource ID.
    azd env set PLAYWRIGHT_SERVICE_RESOURCE_ID "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.LoadTestService/playwrightWorkspaces/<workspace-name>"
    • Playwright Workspace service URL: To get the service URL, open the Playwright Workspace resource and go to the Overview page. Copy the Browser endpoint (it starts with wss://).
    azd env set PLAYWRIGHT_SERVICE_URL "wss://<region>.api.playwright.microsoft.com/playwrightworkspaces/<workspace-id>/browsers"

  2. Provision the resources defined in azure.yaml:

azd provision

This step takes a few minutes and creates the following resources. After provisioning, you see the following message:

SUCCESS: Your application was provisioned in Azure in X minute YY seconds.

Step 2.3: Deploy the agent to Foundry Agent Service

Build and deploy the agent container:

azd deploy

When the agent is deployed, you'll see the following message -

SUCCESS: Your application was deployed to Azure in X minutes YY seconds.

Step 3: Grant the agent access to the Playwright Workspace

The Foundry project's managed identity must have the following role on the Azure Playwright workspace:

Role Purpose
Playwright Workspace Contributor Grants access to create and manage browser sessions
  1. Find the agent's identity object ID in the Foundry portal. Go to Agents, select your agent, and then select Identity.
  2. Assign the role by using Azure CLI:
PRINCIPAL_ID="<project-managed-identity-object-id>"
PWW_RESOURCE_ID="/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.LoadTestService/playwrightWorkspaces/<workspace-name>"

az role assignment create --assignee "$PRINCIPAL_ID" --role "Playwright Workspace Contributor" --scope "$PWW_RESOURCE_ID"

Step 4: Test the agent in the Foundry Playground

  1. Open the Foundry portal and sign in.
  2. Select your project from Recent projects or All projects.
  3. In the left navigation, select Build > Agents.
  4. Select your agent, then select Open in playground.
  5. Enter a prompt such as -
Go to the website finance.yahoo.com, search for MSFT and report the Microsoft stock price.

Step 4: Clean up resources

Delete the resources when you're finished so you stop incurring charges.

azd down

azd lists the resources it deletes and prompts for confirmation. Cleanup takes about 2-5 minutes.

  • Last updated on