Actions

What your agent can do.

An Action is one command against a computer. Choose screen 1, 2, 3, or 4. Screen 1 is used when this is left out.

This list comes from the same contract the API uses. If an Action is missing here, it is missing in the product.

screenshot
Take a picture of one screen
See the screen before you decide what to do next.
Parameters: screen, if_changed, note
Defaults: screen=1, if_changed=true
```
{
  "action": "screenshot",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
click
Click one point
Click an exact pixel on the chosen screen.
Parameters: screen, x, y, button, count, note
Defaults: screen=1, button=left, count=1
```
{
  "action": "click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
double_click
Double click one point
Open a file, folder, or app with two quick clicks.
Parameters: screen, x, y, button, note
Defaults: screen=1, button=left
```
{
  "action": "double_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
triple_click
Triple click one point
Select a full line or use an app's triple click shortcut.
Parameters: screen, x, y, button, note
Defaults: screen=1, button=left
```
{
  "action": "triple_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
right_click
Open a right click menu
Open the menu attached to one point on the screen.
Parameters: screen, x, y, note
Defaults: screen=1
```
{
  "action": "right_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
middle_click
Use the middle mouse button
Use the middle click shortcut at one point.
Parameters: screen, x, y, note
Defaults: screen=1
```
{
  "action": "middle_click",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

mouse_move

Move the pointer

Move the pointer to an exact pixel without clicking.

Parameters: screen, x, y, note

Defaults: screen=1

{
  "action": "mouse_move",
  "screen": 1,
  "x": 640,
  "y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

mouse_down
Hold a mouse button down
Start a mouse hold that a later action can finish.
Parameters: screen, button, note
Defaults: screen=1, button=left
```
{
  "action": "mouse_down",
  "screen": 1,
  "button": "left",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
mouse_up
Release a mouse button
Finish a mouse hold on the chosen screen.
Parameters: screen, button, note
Defaults: screen=1, button=left
```
{
  "action": "mouse_up",
  "screen": 1,
  "button": "left",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
drag
Drag between two points
Hold at one pixel, move to another, then release.
Parameters: screen, start_x, start_y, end_x, end_y, button, note
Defaults: screen=1, button=left
```
{
  "action": "drag",
  "screen": 1,
  "start_x": 100,
  "start_y": 100,
  "end_x": 500,
  "end_y": 400,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
scroll
Scroll the screen
Move the page in one direction by a set amount.
Parameters: screen, direction, amount, x, y, note
Defaults: screen=1, direction=down, amount=3
```
{
  "action": "scroll",
  "screen": 1,
  "direction": "down",
  "amount": 3,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
type
Type into the focused field
Enter text in small chunks so forms receive every character.
Parameters: screen, text, chunk_size, delay_ms, note
Defaults: screen=1, chunk_size=25, delay_ms=75
```
{
  "action": "type",
  "screen": 1,
  "text": "hello",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
key
Press a key or shortcut
Send one shortcut as a combo or as a list of keys.
Parameters: screen, combo, keys, note
Defaults: screen=1
```
{
  "action": "key",
  "screen": 1,
  "combo": "ctrl+shift+t",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

hold_key

Hold a keyboard key

Keep one key down for a set number of milliseconds.

Parameters: screen, key, duration_ms, note

Defaults: screen=1

{
  "action": "hold_key",
  "screen": 1,
  "key": "shift",
  "duration_ms": 250,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

wait
Wait before the next action
Give an app time to finish loading or changing.
Parameters: screen, duration_ms, note
Defaults: screen=1
```
{
  "action": "wait",
  "screen": 1,
  "duration_ms": 1000,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
bash
Run a shell command
Run a command and wait for its output and exit code.
Parameters: screen, command, timeout_ms, note
Defaults: screen=1, timeout_ms=30000
```
{
  "action": "bash",
  "screen": 1,
  "command": "pwd",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
run_background
Start a command in the background
Start a long running command and get its process id.
Parameters: screen, command, note
Defaults: screen=1
```
{
  "action": "run_background",
  "screen": 1,
  "command": "npm run dev",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

open_url

Open a web page

Open a secure web address in the Computer's browser.

Parameters: screen, url, note

Defaults: screen=1

{
  "action": "open_url",
  "screen": 1,
  "url": "https://example.com",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

file_put
Put a file on the Computer
Stream file bytes to a path inside the Computer.
Parameters: screen, path, note
Defaults: screen=1
```
{
  "action": "file_put",
  "screen": 1,
  "path": "/tmp/report.txt"
}
```

file_get

Get a file from the Computer

Stream a file from a path inside the Computer.

Parameters: screen, path, note

Defaults: screen=1

{
  "action": "file_get",
  "screen": 1,
  "path": "/tmp/report.txt",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

clipboard_read
Read the clipboard
Read the text copied on one screen.
Parameters: screen, note
Defaults: screen=1
```
{
  "action": "clipboard_read",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

clipboard_write

Write to the clipboard

Put text on one screen's clipboard.

Parameters: screen, text, note

Defaults: screen=1

{
  "action": "clipboard_write",
  "screen": 1,
  "text": "hello",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

cursor_position
Read the pointer position
Get the pointer's exact pixel on one screen.
Parameters: screen, note
Defaults: screen=1
```
{
  "action": "cursor_position",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```
screen_size
Read the screen size
Get the screen's width and height in pixels.
Parameters: screen, note
Defaults: screen=1
```
{
  "action": "screen_size",
  "screen": 1,
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

launch

Open an app

Start an installed app on one screen, optionally opening a URI.

Parameters: screen, app, uri, note

Defaults: screen=1

{
  "action": "launch",
  "screen": 1,
  "app": "firefox",
  "uri": "https://compuuter.io",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

get_windows
List open windows
See every open window on one screen, optionally filtered by app.
Parameters: screen, app, note
Defaults: screen=1
```
{
  "action": "get_windows",
  "screen": 1,
  "app": "firefox",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}
```

window_title

Read a window title

Get the title of one window by its ID.

Parameters: screen, id, note

Defaults: screen=1

{
  "action": "window_title",
  "screen": 1,
  "id": "4194307",
  "operation_id": "018f5f78-7a36-7d6b-8f93-f0f4c2f8e101"
}

What your agent can do.

Take a picture of one screen

Click one point

Double click one point

Triple click one point

Open a right click menu

Use the middle mouse button

Move the pointer

Hold a mouse button down

Release a mouse button

Drag between two points

Scroll the screen

Type into the focused field

Press a key or shortcut

Hold a keyboard key

Wait before the next action

Run a shell command

Start a command in the background

Open a web page

Put a file on the Computer

Get a file from the Computer

Read the clipboard

Write to the clipboard

Read the pointer position

Read the screen size

Open an app

List open windows

Read a window title