Read Console

Developer guide

Build against action scripts without guessing how the runtime works

This page explains the real system flow for action scripts: how to create them in the console, how to edit steps, how bindings work, how to call the run endpoint, and how to send text or multi-image inputs so each paste lands on the intended field.

Works for both console operators and backend or AI callers using personal keys

Explains the difference between static step values and runtime-bound inputs

Includes concrete payload examples for text, text-list, artifact, and artifact-list

Open scripts View pricing

Bound paste step

{
  "kind": "key",
  "action": "paste",
  "modifiers": ["control"],
  "binding": {
    "slot": "title",
    "sourceType": "text"
  }
}

Model

How an action script is evaluated

A script is an ordered list of steps. Each step has its own delay and input shape. If a step does not define `binding`, it uses the value already stored inside that step.

Static text

Use for fixed values that should always be typed exactly as recorded.

Static paste

Only sends the paste key. The target device will paste whatever is already in its clipboard at that moment.

Bound text

The step reads its value from the `inputs` object sent at run time.

Bound paste

The system pushes clipboard content to the target device first, then sends paste. This is the reliable option for business-critical data.

Static paste

{
  "kind": "key",
  "action": "paste",
  "modifiers": ["control"]
}

Bound image paste

{
  "kind": "key",
  "action": "paste",
  "modifiers": ["control"],
  "binding": {
    "slot": "productImages",
    "sourceType": "artifact-list",
    "index": 0,
    "contentType": "image/png"
  }
}

Create

How to create a script

The right way to create a script is to record it from the real control surface so the click order, delays, and target fields match the destination workflow exactly.

Console flow

Go to Devices > choose a device > Screen

Open an interactive control session and enter full-screen if the workflow is long

Click Record action

Perform the real flow: click, type, paste, navigate, or wait

Click Stop & save and give the script a clear name such as Post product to Shopee

Implementation note

If the script should receive different values on each run, do not leave the critical steps static. Open the step editor and add bindings explicitly.

Edit

How to edit a script

Settings > Action scripts exposes the raw JSON for each step. This is the place to convert static steps into runtime-bound steps, reorder steps, or tighten the run sequence for automation.

Use Edit steps to open the step editor for the whole script

`kind: "text"` can use a static `text` value or a runtime `binding`

`kind: "key"` with `action: "paste"` can use a binding so the platform injects clipboard content before paste

List bindings such as `text-list` and `artifact-list` must always include `index`

Supported steps

Every action-script step kind supported right now

The runtime currently supports the seven step families below. Each one maps to a distinct execution behavior, so choose the smallest step that matches the target interaction.

Coordinate values such as `x`, `y`, `startX`, `startY`, `endX`, and `endY` are normalized from 0 to 1 against the remote display viewport, not fixed pixels.

tap

Use for a single click or tap. Supports `button`, `clickCount`, and `modifiers`. Good for buttons, inputs, checkboxes, and double-click flows.

{
  "kind": "tap",
  "x": 0.52,
  "y": 0.33,
  "button": "left",
  "clickCount": 1,
  "modifiers": []
}

longPress

Press and hold one point. Good for context menus, hold-to-select gestures, or controls that need deliberate press duration.

{
  "kind": "longPress",
  "x": 0.48,
  "y": 0.62,
  "durationMs": 900,
  "modifiers": []
}

swipe

Drag from a start point to an end point. Works for mobile swipe flows and some deterministic drag gestures when live pointer tracking is unnecessary.

{
  "kind": "swipe",
  "startX": 0.82,
  "startY": 0.78,
  "endX": 0.18,
  "endY": 0.78,
  "durationMs": 320
}

scroll

Scroll at a specific location using `deltaX` and `deltaY` values between -1 and 1. Good for feeds, tables, drawers, and long forms.

{
  "kind": "scroll",
  "x": 0.5,
  "y": 0.5,
  "deltaX": 0,
  "deltaY": -0.65,
  "durationMs": 250
}

global

System-level navigation. Currently supports `back`, `home`, and `recents`. Best for Android navigation and shell-level movement where available.

{
  "kind": "global",
  "action": "back"
}

text

Type text into the currently focused field. Can use static `text` or runtime `binding`. Good for titles, captions, OTPs, search terms, and IDs.

{
  "kind": "text",
  "binding": {
    "slot": "caption",
    "sourceType": "text"
  }
}

key

Send a key action such as `enter`, `tab`, `escape`, `backspace`, `delete`, `arrowUp`, `pageDown`, or `paste`. Binding is allowed only for `paste`.

{
  "kind": "key",
  "action": "enter",
  "modifiers": []
}

Bindings

How to pass data into a script

Runtime data is sent inside `inputs`, where each named slot maps to one input type. Slot names should represent the business field directly: `title`, `caption`, `hashtags`, `productImages`.

`text`

One text value such as a title or caption.

`text-list`

An ordered list of text values such as hashtags or repeated text targets.

`artifact`

One file-like clipboard payload such as a hero image or PDF.

`artifact-list`

Several ordered files such as productImages[0], productImages[1], productImages[2].

Do not reuse one slot name across incompatible data types. For example, using `payload` once as text and elsewhere as artifact-list will be rejected at runtime.

Run API

How to run a script from an external system

External callers use a personal key with the `action-scripts:run` scope, then POST to the external run endpoint with a target `deviceId` and the `inputs` required by that script.

Run sequence

Create a personal key in Settings > Action scripts

Enable the `action-scripts:run` scope

Call `POST /v1/external/action-scripts/{actionScriptId}/runs`

Send `Authorization: Bearer <personal-key>`

Send `deviceId`, optional `paceMultiplier`, and the matching `inputs`

If the script has no bindings, `inputs` is optional. If the script has bindings and a required slot is missing or typed incorrectly, the run is rejected immediately.

External run payload

POST /v1/external/action-scripts/{actionScriptId}/runs
Authorization: Bearer <personal-key>
Content-Type: application/json

{
  "deviceId": "<deviceId>",
  "paceMultiplier": 1,
  "inputs": {
    "title": {
      "kind": "text",
      "value": "Nike Air Zoom Pegasus 41"
    },
    "caption": {
      "kind": "text",
      "value": "Lightweight daily running shoe with responsive cushioning."
    },
    "hashtags": {
      "kind": "text-list",
      "items": ["#nike", "#running", "#pegasus41"]
    },
    "productImages": {
      "kind": "artifact-list",
      "items": [
        {
          "contentType": "image/png",
          "artifact": {
            "bucket": "read-artifacts",
            "objectKey": "clipboard/product-images/item-1",
            "fileName": "item-1.png",
            "byteCount": 1024,
            "sha256": "<sha256-1>",
            "downloadUrl": "https://example.com/item-1.png",
            "previewUrl": "https://example.com/preview/item-1.png",
            "expiresAt": "2026-07-04T00:00:00.000Z"
          },
          "preview": "product image 1"
        }
      ]
    }
  }
}

Artifacts

What to know about image and file inputs

The external run endpoint supports `artifact` and `artifact-list`, but those artifact descriptors must already exist. Today the personal-key run endpoint executes scripts; it does not mint upload URLs for new artifacts by itself.

A signed-in web console can create clipboard artifacts and upload files before running the script

A trusted internal service can prepare artifacts through the authenticated console APIs, then pass their descriptors into `inputs`

If an external caller only has a personal key and no artifact-preparation path, text inputs work immediately but image flows require an additional artifact staging step

Reliability

How to keep AI and backend callers accurate

Reliable automation depends on deterministic scripts and explicit runtime inputs. Do not depend on remote state that the caller cannot prove or reset.

Bind every business-critical value such as title, caption, hashtags, and product images

Use `artifact-list` with explicit indexes for multi-image workflows

Avoid relying on the target device clipboard for important data unless that is intentional

Use clear semantic slot names instead of vague names such as `data1` or `payload`

After editing a script, validate it once on a real device before handing it to another AI or backend workflow

Loading control center

Gathering device status, approvals, and operator signals for the main workspace.

POST /v1/external/action-scripts/{actionScriptId}/runs Authorization: Bearer <personal-key> Content-Type: application/json { "deviceId": "<deviceId>", "paceMultiplier": 1, "inputs": { "title": { "kind": "text", "value": "Nike Air Zoom Pegasus 41" }, "caption": { "kind": "text", "value": "Lightweight daily running shoe with responsive cushioning." }, "hashtags": { "kind": "text-list", "items": ["#nike", "#running", "#pegasus41"] }, "productImages": { "kind": "artifact-list", "items": [ { "contentType": "image/png", "artifact": { "bucket": "read-artifacts", "objectKey": "clipboard/product-images/item-1", "fileName": "item-1.png", "byteCount": 1024, "sha256": "<sha256-1>", "downloadUrl": "https://example.com/item-1.png", "previewUrl": "https://example.com/preview/item-1.png", "expiresAt": "2026-07-04T00:00:00.000Z" }, "preview": "product image 1" } ] } } }