Camera
Third-person camera
Wrist camera
Live diagnostics
- Closure: requested / measured
- --
- Mean pad effort (N, uncalibrated)
- --
- Pad contacts / holding
- --
- Received age / interval (ms)
- --
- Object XYZ (cm)
- --
- Object velocity (cm/s)
- --
- Active command
- --
- Gateway fault
- --
| Simulation (s) | Pad effort (N) | Contacts | Object Z (cm) | Feedback |
|---|
Latest telemetry JSON
No sample
Checkmate
Kinematic move demoRun details
No transfer yet
| a | b | c | d | e | f | g | h |
|---|
Waiting for assigned object feedback
Top-down XY projection / Read-onlyProjected positions, not placement or chess-legality validation.
Jog control
ObservingAdvanced session control
- 1Base
- 2Shoulder
- 3Elbow
- 4Wrist 1
- 5Wrist 2
- 6Wrist 3
Gripper
Environment
Python API: observe_state() and observe_environment()
Read Environment data
In the development starter, run connect.py first. This example reads the arm assigned in connection.private.json without acquiring control or moving it.
GatewayClient.observe_state(arm_id) returns object origins and arm joint locations in centimeters under data["world"]. GatewayClient.observe_environment(arm_id) returns unreal_truth and sensor_truth snapshots with positions in meters. Sensor availability and freshness are reported separately; missing sensor data is not replaced with Unreal Truth.
Unreal Truth
Arm locations
Arm feedback unavailable
| Object ID | X | Y | Z | Physics |
|---|
Sensor Truth
| Track ID | Position (m) | Uncertainty (m) |
|---|
Sensor evidence
No observation
API
Read arm and scene feedback, command joints and the gripper, and inspect or manage assigned objects. Python skills use these same APIs from your laptop.
Download Visual Studio Code starter- Selected arm
- --
- Skill runtime
- Local Python client
- Server Python uploads
- Disabled
- Object source
- Simulation ground truth
Read scene feedback
After running connect.py in the starter, this reads the selected arm without taking control. Object positions are actor origins in centimeters; manipulation destinations use meters. The environment API also exposes Sensor Truth and its availability, freshness and qualification.
Move a joint from Python
Release browser control first. This example requests exclusive control, reads the current six angles, changes only joint 1, waits for completion and reads feedback. It leaves the gripper unchanged. Only run a target after inspecting the arm and path; joint limits are not collision avoidance.
from robot_gateway_client import GatewayClient, load_connection
from robot_skill_session import ArmSession
connection = load_connection("connection.private.json")
client = GatewayClient.from_config(connection)
if input("Move joint 1 to 5 degrees? Type MOVE: ") == "MOVE":
with ArmSession(connection["arm_id"], "my-joint-example", client=client) as arm:
target = list(arm.state()["joints_deg"])
target[0] = 5.0
result = arm.move_joints(target, timeout_s=15)
print(result)
print(arm.state())
The starter's manual_control.py --joint 1 --degrees 5 previews this target without motion. Append --allow-motion only when ready to execute.
Calls, Inputs and Results
View tool schemas (JSON)Each entry below includes a Python SDK example and the complete HTTP request. tools.json supplies the input schemas, result descriptions and permissions. Availability is configuration, not a live readiness test.
Start here: download the Visual Studio Code starter, install it, then run python connect.py. Python samples run in that starter's virtual environment. They load your selected arm and temporary API credential from connection.private.json; browser sign-in alone does not authenticate your code.
Hosted API base URL: Loading selected portal address. On your laptop, always use connection["api_url"], especially if viewing an editable website on a different local port. Direct HTTP clients send X-Robot-Access-Key with the temporary API key, not your portal sign-in key. Do not send a browser Origin header, put keys in URLs, or include secrets in shared examples.
Direct HTTP: control leases and command completion
Observation calls need no lease. For a command, release browser control, call leases/acquire, and use the returned result.token in Authorization: Bearer .... This lease token is separate from the API access key. If acquisition is rejected or queued, do not submit a command without a granted token.
Submit the tool's full command envelope to commands/submit with a new UUID as command_id. Poll commands/get using that same ID until the status is terminal: completed, failed, cancelled, expired, or interrupted. A queued/running submission is not success. Renew a 30-second lease before it expires, for example every 10 seconds, and release it when finished.
Every HTTP success is wrapped as {"ok": true, "result": ...}. Observation data is in result.data; command records include status and, when available, result with final action data. On an HTTP error, inspect error_code and error. Never automatically resubmit an uncertain command or open the gripper during cleanup.
Call an HTTP endpoint directly from Python (no SDK request helper)
This read-only example uses the standard library HTTP client. It does not follow redirects or disable certificate validation. Change the method, endpoint suffix and JSON body to match an entry below; leased requests additionally require the lease token header.
import http.client
import json
from urllib.parse import urlsplit
from robot_gateway_client import GatewayClient, load_connection
connection = load_connection("connection.private.json")
GatewayClient.from_config(connection)
endpoint = urlsplit(connection["api_url"])
transport = http.client.HTTPSConnection if endpoint.scheme == "https" else http.client.HTTPConnection
http = transport(endpoint.hostname, endpoint.port, timeout=10)
try:
http.request("POST", endpoint.path + "/observations/state",
body=json.dumps({"arm_id": connection["arm_id"]}),
headers={"Content-Type": "application/json",
"X-Robot-Access-Key": connection["access_key"]})
response = http.getresponse()
payload = json.loads(response.read(65536))
if response.status != 200 or not payload.get("ok"):
raise RuntimeError(payload.get("error", "HTTP request failed"))
print(payload["result"])
finally:
http.close()
Read object planning metadata
describe_object reads an assigned object's pose, bounds and mass, nearby obstacles, reference joint geometry, and gripper limits. A planner uses this to assess a grasp and route. It does not move, pick up, scan with a camera, or certify that an object is safe to grasp.
Requires the velocity arm and native metadata service. The request briefly acquires control when needed, then releases it; joints and gripper remain unchanged.
Metadata result (JSON)
Select an object and read its metadata.
Physical test objects
Velocity armFrom the starter terminal: python objects.py spawn previews a box without changing the scene. After checking the workspace, python objects.py spawn --allow-scene-change creates it. Both the browser and Python API enforce the same arm faults, empty-gripper requirement and object limits.
- Placement
- Clear area on your assigned board
- Object limit
- 8 per velocity arm
- Persistence
- Current simulation session
No object operation
Additional platform capabilities
| Capability / API | Status | Requirements and limits |
|---|
Skills
Python / Your laptop1. Understand a skill
A skill is an abstracted ability that accomplishes a task. It can call several other skills: Move item combines Pick up, Move and Put down. A skill defines the information it needs, the actions it can request, and the feedback that proves success or tells it to stop.
Sensors and a shared world view
Sensors populate the world view. Skills use that world view. Robot adapters own kinematics and motion.
Cameras, depth sensors and other sensors publish structured estimates into the Sensor Truth view. Task skills consume those estimates, or Unreal Truth in simulation, as feedback. They do not need to know which camera produced an object's location.
This boundary lets you add or swap sensors while reusing skills, provided their adapters supply the required fields, coordinate frames, freshness and confidence. Missing evidence must stop a skill; a different sensor is not automatically equivalent.
Current manipulation uses simulation truth. Sensor-populated world-view control is not yet qualified. Define observable success, force/lift/slip checks and a time budget. The reference allows 15 seconds per component; setup and planning are separate.
Design principles and author checklist2. Follow the feedback loop
- Your laptop → APIRead the world view
Get arm state, object locations and available sensor estimates.
- Your laptop / PythonRun a skill and its subskills
Query the view again for required object pose, contact or destination data.
- API → ARCADE armRequest an action
The adapter plans motion; the gateway checks permission and exclusive control.
- Arm and sensors → APIReturn measured feedback
As the arm moves, telemetry and sensor producers update the world view.
AI is optional and belongs inside your Python skill. To use it, implement a call to your own AI endpoint for assistance. ARCADE does not add an AI call automatically. Keep endpoint keys in local environment variables, validate model output, and retain the same feedback checks and motion permissions.
3. Compose Move item
Move item is the display name; its Python skill ID remains move_object. It accepts an object ID, destination actor-origin XYZ in world meters, target rotation and a grip-force ceiling. It obtains planning metadata and shares one checked plan, one control session and continuous grasp tracking across the components.
- Pick up (
pick_up)Needs the object identity and pose, geometry/mass, gripper limits, force ceiling and planned approach. Opens the empty gripper, approaches, closes, checks bilateral contact and the correct held object, records a grasp reference, and verifies actual lift.
- Move (
move)Needs the successful pickup context, original grasp reference, destination, clearance and checked carry route. Moves the already-held object above its destination while checking contact, force, orientation and relative slip. This is not a general joint jog.
- Put down (
put_down)Needs the successful carry context, placement route, support height, target pose and tolerances. Lowers, opens the gripper, verifies release before withdrawing, then checks stable final position, orientation and low velocity.
These components are implemented but experimental and simulation-only. They must run in order within the same transfer context, not as independent commands. Individual simulation runs passed and another failed placement; repeatability and hardware validation remain unproven.
Move item contract and evidenceSkill catalog
Implementation labels describe source availability, not live readiness or physical safety certification.
Composed skills and roadmap
Chess examples have a separate contract
Chess square transfer uses the authored board and kinematic dispatcher. The older "Pick and place assigned piece" script uses that same chess path, not the Pick up, Move and Put down composition. It is no longer a separate catalog entry. Move item uses the velocity arm and destination coordinates, not a chess square. Chess transfers support only ordinary non-capturing moves to empty squares; captures, castling, promotion, en passant, board-state updates and chess legality belong to a separate chess layer.
4. Download one development workspace
Download Visual Studio Code starterInstall Visual Studio Code, Python 3.11 or 3.12 (recommended), and the VS Code Python extension. Extract the ZIP into one folder and open that root folder in VS Code. No local Unreal, ROS or full repository checkout is required.
README.md,.vscode/- Setup, contents, test tasks and read-only debug configurations.
sdk/,tools.json- Python client and the API input/result reference.
skills/- Move item (
move_object), Pick up, Move, Put down, object report and the compatible pick-and-place implementation. connect.py,scene_data.py,manual_control.py- Local sign-in, scene feedback, and manual joint control;
watch_state.pyrecords continuing feedback. website/- Editable ARCADE website, local connector, Three.js source and its setup README.
.github/,tests/- GitHub Copilot instructions and skill-authoring workflow, plus offline example tests.
5. Connect your laptop and read feedback
Selected arm: --. The ZIP records this arm and the portal address, not your credentials. For another laptop, download from the operator-approved HTTPS address; 127.0.0.1 means that laptop, not the ARCADE server.
In VS Code, run Python: Create Environment, choose Venv, then open a new terminal using that environment. From the extracted root folder, run:
connect.py prompts for the same portal access key you use to sign in here. Type it only into your local terminal's hidden prompt. It creates connection.private.json automatically and reports "connected": true when authenticated feedback is available. scene_data.py then prints arm, object and environment data without moving anything.
The Python API credential is selected-arm scoped and lasts up to one hour. Its sign-in session is separate from this browser. Run python connect.py --disconnect to revoke the Python session and its control; the gripper stays unchanged. For an expired session, disconnect and run connect.py again.
The generated private file contains authentication secrets. Keep it local and out of commits, screenshots and AI prompts. No separate credential download or raw robot port is needed.
Try manual joint control
python manual_control.py --joint 1 --degrees 5
This is a preview. Release browser control, inspect the arm and path, then append --allow-motion to execute. The shows the underlying Python calls. Do not automatically retry a failed or uncertain motion.
6. Develop, test and optionally use Copilot
Start with skills/object_report/. Define inputs, observable success and failure, implement your Python skill, and run python -m pytest -q offline. For manipulation, inspect metadata and try a plan-only run before separately authorizing any movement.
python skills/move_object/skill.py --object YOUR_OBJECT_ID --plan-only
Use an exact ID from scene feedback. To specify a destination, add --to-m X Y Z with world meters. Pick up, Move and Put down consume the same MoveObjectExecution context; their source and READMEs show how to compose them.
For GitHub Copilot, sign in to an account with Copilot access and open the extracted root, including .github/. In Chat, invoke /arcade-skill-author with a task such as "Design a skill and offline failure tests; do not contact or move a robot." This authoring workflow is optional and is separate from calling AI at skill runtime.
Run the included website locally
From the starter root, use the same virtual environment. The connector chooses a free loopback port and prints the local URL. Sign in there with your portal access key. The website starts read-only; Python client permissions are independent.
python -m pip install -r website/requirements.txt python website/serve.py
Edit website/public/; see website/README.md for media networking, Three.js builds and explicit motion authorization.
Workspace setup checks
Setup test results
No setup tests run
Read-only checks
These checks read the selected arm and browser streams. They do not move joints, open the gripper or reset the workspace. A telemetry pass is not a load-bearing grip or physical safety certification.
Chess piece pickup tests
Selected armAcceptance: verified pickup and intentional release. Falling after release is allowed. Knight grasp failures are listed as allowed, not passed.
No pickup test run
No results yet
Desktop test review
Adapted from the Python Jog tests: payload validation becomes Joint input limits; integration checks become assigned feedback, object, grip and preview checks. Global process status, full pose selftest, ROS/Unreal recovery and arm comparison remain operator-only because they inspect shared processes, home shared arms or interrupt other users.
ROS Monitor
Read-only / Developer- Selected arm
- --
- ROS feedback
- Not configured
- Expected bridge node
- --
- Collection
- ROS 2 subscriptions
ROS traffic only. Normal browser control uses the direct gateway and does not publish ROS command topics. Topic mappings are operator-configured; they are not proof of source identity. Camera rows contain metadata, not an image feed.
| Topic / Type | Health | Hz | Age (s) | Messages | Publisher nodes | Subscriber nodes | Latest payload |
|---|
Commanded and actual joints
Waiting for both named joint messages. No direct-gateway fallback.
| Joint | Command (deg) | Actual (deg) | Actual - command (deg) |
|---|
Admin view
Host health
Memory: --
Previews: --
GPU: waiting
Control admission
Waiting for control
Waiting for snapshot
Admission capacity
- Occupied workspaces
- -- / -- limit
- Advanced control
- --
Control holders
Admission policy
Control requests are FIFO. Lowering admission does not interrupt existing execution. GPU counters are display-only; resource forecasts are estimates, not guarantees.
Executing and pending
Waiting for snapshot
Low-rate previews
For the selected workspace only. Uses a separate FIFO queue; FPS, CPU, RAM and fresh telemetry checks still apply.
Arm inventory
| Arm / access assignment | Observed motion | Preview | Advanced control owner | Execution | Shared note |
|---|
No matching arms
Motion is derived from existing feedback; stale or unobserved arms show Unknown. This view opens no camera streams. Guest Windows counters do not prove exclusive physical GPU utilization.