Actions

What your agent can do.

An Action is one command against a computer. Choose screen 1, 2, 3, or 4. Screen 1 is used when this is left out.

This list comes from the same contract the API uses. If an Action is missing here, it is missing in the product.

actions
Run ordered actions and see the screen
Perform up to 24 ordered desktop actions and return the resulting screen. Batch only predictable actions; stop before an outcome you need to inspect. Never redo completed or uncertain steps.
Parameters: screen, actions, observe, settle_ms, max_width, frame_hash, note
Defaults: screen=1, observe=true
```
{
  "action": "actions",
  "actions": [
    {
      "action": "click",
      "x": 640,
      "y": 400
    },
    {
      "action": "type",
      "text": "hello"
    },
    {
      "action": "key",
      "combo": "Return"
    }
  ]
}
```
screenshot
Take a picture of one screen
See the screen before you decide what to do next.
Parameters: screen, if_changed, frame_hash, format, max_width, note
Defaults: screen=1, if_changed=false
```
{
  "action": "screenshot",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
click
Click one point
Click an exact pixel on the chosen screen.
Parameters: screen, x, y, button, count, observe, max_width, frame_hash, note
Defaults: screen=1, button=left, count=1
```
{
  "action": "click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
double_click
Double click one point
Open a file, folder, or app with two quick clicks.
Parameters: screen, x, y, button, observe, max_width, frame_hash, note
Defaults: screen=1, button=left
```
{
  "action": "double_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
triple_click
Triple click one point
Select a full line or use an app's triple click shortcut.
Parameters: screen, x, y, button, observe, max_width, frame_hash, note
Defaults: screen=1, button=left
```
{
  "action": "triple_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
right_click
Open a right click menu
Open the menu attached to one point on the screen.
Parameters: screen, x, y, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "right_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
middle_click
Use the middle mouse button
Use the middle click shortcut at one point.
Parameters: screen, x, y, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "middle_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
mouse_move
Move the pointer
Move the pointer to an exact pixel without clicking.
Parameters: screen, x, y, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "mouse_move",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
mouse_down
Hold a mouse button down
Start a mouse hold that a later action can finish.
Parameters: screen, button, observe, max_width, frame_hash, note
Defaults: screen=1, button=left
```
{
  "action": "mouse_down",
  "screen": 1,
  "button": "left",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
mouse_up
Release a mouse button
Finish a mouse hold on the chosen screen.
Parameters: screen, button, observe, max_width, frame_hash, note
Defaults: screen=1, button=left
```
{
  "action": "mouse_up",
  "screen": 1,
  "button": "left",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
drag
Drag between two points
Hold at one pixel, move to another, then release.
Parameters: screen, start_x, start_y, end_x, end_y, button, observe, max_width, frame_hash, note
Defaults: screen=1, button=left
```
{
  "action": "drag",
  "screen": 1,
  "start_x": 100,
  "start_y": 100,
  "end_x": 500,
  "end_y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
scroll
Scroll the screen
Move the page in one direction by a set amount.
Parameters: screen, direction, amount, x, y, observe, max_width, frame_hash, note
Defaults: screen=1, direction=down, amount=3
```
{
  "action": "scroll",
  "screen": 1,
  "direction": "down",
  "amount": 3,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
type
Type into the focused field
Enter text in small chunks so forms receive every character.
Parameters: screen, text, chunk_size, delay_ms, observe, max_width, frame_hash, note
Defaults: screen=1, chunk_size=25, delay_ms=75
```
{
  "action": "type",
  "screen": 1,
  "text": "hello",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
key
Press a key or shortcut
Send one shortcut as a combo or as a list of keys.
Parameters: screen, combo, keys, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "key",
  "screen": 1,
  "combo": "ctrl+shift+t",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
hold_key
Hold a keyboard key
Keep one key down for a set number of milliseconds.
Parameters: screen, key, duration_ms, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "hold_key",
  "screen": 1,
  "key": "shift",
  "duration_ms": 250,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
wait
Wait before the next action
Give an app time to finish loading or changing.
Parameters: screen, duration_ms, note
Defaults: screen=1
```
{
  "action": "wait",
  "screen": 1,
  "duration_ms": 1000,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
bash
Run a shell command
Run a command and wait for its output and exit code.
Parameters: screen, command, timeout_ms, note
Defaults: screen=1, timeout_ms=30000
```
{
  "action": "bash",
  "screen": 1,
  "command": "pwd",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
run_background
Start a command in the background
Start a command and get its process id. To get a preview_url, print PORT=8123, port 8123, or :8123 in startup output. Replace 8123 with your server's port. Only the first 64 KiB of startup output is checked.
Parameters: screen, command, note
Defaults: screen=1
```
{
  "action": "run_background",
  "screen": 1,
  "command": "npm run dev",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
stop_background
Stop a background command
Stop a process started by run_background on this Compuuter.
Parameters: screen, pid, note
Defaults: screen=1
```
{
  "action": "stop_background",
  "screen": 1,
  "pid": 91,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
open_url
Open a web page
Open a secure web address in the Compuuter's browser.
Parameters: screen, url, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "open_url",
  "screen": 1,
  "url": "https://example.com",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
file_put
Put a file on the Compuuter
Stream file bytes to a path inside the Compuuter.
Parameters: screen, path, note
Defaults: screen=1
```
{
  "action": "file_put",
  "screen": 1,
  "path": "/tmp/report.txt"
}
```
file_get
Get a file from the Compuuter
Stream a file from a path inside the Compuuter.
Parameters: screen, path, note
Defaults: screen=1
```
{
  "action": "file_get",
  "screen": 1,
  "path": "/tmp/report.txt",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
clipboard_read
Read the clipboard
Read the text copied on one screen.
Parameters: screen, note
Defaults: screen=1
```
{
  "action": "clipboard_read",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
clipboard_write
Write to the clipboard
Put text on one screen's clipboard.
Parameters: screen, text, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "clipboard_write",
  "screen": 1,
  "text": "hello",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
cursor_position
Read the pointer position
Get the pointer's exact pixel on one screen.
Parameters: screen, note
Defaults: screen=1
```
{
  "action": "cursor_position",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
screen_size
Read the screen size
Get the screen's width and height in pixels.
Parameters: screen, note
Defaults: screen=1
```
{
  "action": "screen_size",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
launch
Open an app
Start an installed app on one screen, optionally opening a URI.
Parameters: screen, app, uri, observe, max_width, frame_hash, note
Defaults: screen=1
```
{
  "action": "launch",
  "screen": 1,
  "app": "firefox",
  "uri": "https://compuuter.io",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
get_windows
List open windows
See every open window on one screen, optionally filtered by app.
Parameters: screen, app, note
Defaults: screen=1
```
{
  "action": "get_windows",
  "screen": 1,
  "app": "firefox",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

window_title

Read a window title

Get the title of one window by its ID.

Parameters: screen, id, note

Defaults: screen=1

{
  "action": "window_title",
  "screen": 1,
  "id": "4194307",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

act_on
Click or type on a control by its name
Act on a control the Screen already named, taken from a snapshot, instead of at a pixel you guessed. Prefer this on ordinary apps; use the picture for games, drawing canvases, and screens with no tree. A target that matches more than one control is refused with the names it matched.
Parameters: screen, target, node_id, act, text, observe, max_width, frame_hash, note
Defaults: screen=1, act=click, observe=true
```
{
  "action": "act_on",
  "screen": 1,
  "target": "Home",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

What your agent can do.

Run ordered actions and see the screen

Take a picture of one screen

Click one point

Double click one point

Triple click one point

Open a right click menu

Use the middle mouse button

Move the pointer

Hold a mouse button down

Release a mouse button

Drag between two points

Scroll the screen

Type into the focused field

Press a key or shortcut

Hold a keyboard key

Wait before the next action

Run a shell command

Start a command in the background

Stop a background command

Open a web page

Put a file on the Compuuter

Get a file from the Compuuter

Read the clipboard

Write to the clipboard

Read the pointer position

Read the screen size

Open an app

List open windows

Read a window title

Click or type on a control by its name