Read the page. Then act on it.

Every web page has things you can act on: boxes, links, buttons. The page tools hand your agent that list. Each one gets a short tag like e1. Your agent then acts by tag, so a click lands on the button it meant.

Three tools do the whole job: browser_navigate, browser_snapshot, and browser_act. They run on the Chrome already open on that screen.

A sign-in, start to finish

1. 01

   Open the page

   Send the address. It has to start with https, and it can't carry a password.

   { "command": "navigate", "url": "https://example.com/sign-in" }

2. 02

   Read what's there

   You get the address, the title, the words on the page, and up to 80 things you can act on. A password box never sends its value back.

   {
     "ok": true,
     "url": "https://example.com/sign-in",
     "title": "Sign in",
     "elements": [
       { "ref": "e1", "role": "textbox", "name": "Email", "value": "" },
       { "ref": "e2", "role": "textbox", "name": "Password" },
       { "ref": "e3", "role": "button", "name": "Sign in" }
     ],
     "tree": "Sign in\n- textbox \"Email\" [e1]\n- textbox \"Password\" [e2]\n- button \"Sign in\" [e3]"
   }

3. 03

   Act by tag

   Up to 24 steps in one call. completed comes back saying how many landed.

   {
     "command": "act",
     "actions": [
       { "kind": "fill", "ref": "e1", "text": "ada@example.com" },
       { "kind": "click", "ref": "e3" }
     ]
   }

Two rules that keep it honest

A tag only lasts until the page changes. Read the page again after anything moves. An old tag is refused before a single click goes out, so a stale plan can't half-run.

A refusal names its way out. When the page can't be driven this way, the answer carries fallback: computer_act. That means finish the job with the desktop Actions: a screenshot, then clicks and keys. uncertain: true means a step may already have landed, so read the page before you repeat it.

{
  "ok": false,
  "completed": 1,
  "uncertain": true,
  "error": "Stale element ref. Take a fresh snapshot.",
  "fallback": "computer_act"
}

The rest of what your agent can do lives on Actions.