Hand Control for developers and AI agents.

How an agent reaches Hand Control

What an agent can and cannot do

All of this happens on your PC. The connector talks to Hand Control over 127.0.0.1 and nothing leaves the machine. Anything the agent then does with what it read, such as sending it to its own model, is up to that agent and its app.

Get Hand Control

Install Hand Control, then turn on the agent API in the Agents tab as described in step 1.

Back to the contents

Connect Claude in three steps

1Step 1: Turn on the agent API

In Hand Control, open the Agents tab and tick Allow local programs to connect (agent API). Tick the two permission boxes only if you want an agent to be able to do those things.

2Step 2: Connect your agent

Hand Control installs a connector, HandControlMCP.exe, next to the app. It speaks the Model Context Protocol, both the current revision (2026-07-28) and the older handshake-based revisions, so older and newer clients both work. It finds the port and the access key by itself.

The Agents tab has two buttons that copy the exact text for your install:

Claude Code

Copy for Claude Code, then paste it into a terminal. It looks like this:

Terminal

claude mcp add --scope user --transport stdio hand-control -- "C:\Users\<you>\AppData\Local\Programs\Hand Control\HandControlMCP.exe"

--scope user makes it available in every project. Check it with claude mcp list.

Claude Desktop

Copy for Claude Desktop. In Claude Desktop, open Settings > Developer > Edit Config, which opens %APPDATA%\Claude\claude_desktop_config.json. Merge the copied block into mcpServers:

{
  "mcpServers": {
    "hand-control": {
      "command": "C:\\Users\\<you>\\AppData\\Local\\Programs\\Hand Control\\HandControlMCP.exe",
      "args": []
    }
  }
}

Quit Claude Desktop completely and start it again.

Any other MCP client

Run HandControlMCP.exe as a stdio server with no arguments. It reads api_port from %APPDATA%\Hand Control\settings.json when it starts, so restart your agent after changing the port; --port <n> (1024–65535) forces a port; any other value is ignored. --version prints the version. --selftest checks the exe and exits.

Without MCP

Scripts and your own programs can use the HTTP API directly; see the local API reference.

3Step 3: Try it

Ask your agent, for example:

• “Is Hand Control on? Which program is it set up for?”
• “Wait until I make the rotate gesture, then tell me.”
• “Switch Hand Control to Fusion 360.” (needs the control permission)
• “Look through the Hand Control camera: is my hand in view?” (needs the camera permission)

Troubleshooting

To shut every program out at once, untick the box in step 1. New key stops any program that kept a copy of the old key; a program that reads the key file each time, such as this connector, keeps working.

Back to the contents

Two-hand gestures in 3D programs

Hand Control drives Windows with ordinary mouse input, so most desktop programs work with it: CAD, modelling tools, browsers and slide shows. You do not install a plug-in into the program. Programs that run as administrator need Hand Control to run as administrator too (see below), and games that read raw mouse input may ignore the pointer movement.

The one-hand gestures (point, pinch to click and right-click, V sign to scroll; or the finger-tap style) behave the same everywhere. The two-hand gestures (rotate, pan, zoom) send the view controls of the program you choose in Gestures > Two-hand gestures drive.

Supported 3D programs

The bindings were checked against the sources above on 26 Sep 2026 (Fusion 360 again on 28 Sep 2026). If you have changed a program’s mouse settings, pick the profile whose controls match yours.

How the two-hand gestures work

Safety behaviour, which is the same in every profile:

• Everything held is released when a hand leaves view, when the camera stalls, when the camera or Mirror changes, and when you are no longer recognised with Only me on.
• Switching the profile in the middle of a gesture releases everything and takes the new controls.
• If Windows refuses to release a button (it can when the program in front runs as administrator), Hand Control sends nothing else until that release goes through.
• Ctrl+Alt+H switches control on or off while the camera is running, whatever your hands are doing.

Programs that run as administrator

Windows does not let a normal program send input to a program running as administrator (User Interface Privilege Isolation). If Hand Control does nothing in one program, check whether that program was started “as administrator”. Either start it normally, or start Hand Control as administrator too.

Programs without a profile

Most 3D programs let you choose mouse controls. Pick the preset that matches one of the profiles above; the SolidWorks or Blender styles are the most common. The profile is saved in %APPDATA%\Hand Control\settings.json as app_profile.

Connecting other software

Back to the contents

The local API

Hand Control can let other programs on the same PC see what it sees: whether it is running, the hands it is following (21 landmarks each), and every gesture and input event. With the user’s permission, a program can also turn control on or off, choose the 3D program profile, or take a camera snapshot.

The API cannot start any mouse or keyboard input: only the user’s hands do that. Turning control off, or switching the 3D program in the middle of a two-hand gesture, lets go of what the hands are holding (a program switch then takes the new program’s buttons for the same gesture).

If you are connecting an AI agent (Claude Code, Claude Desktop or any other MCP client), you do not need this page: use the MCP connector described in Connect Claude. This page is for scripts and your own programs.

Turning it on

The API is off until the user turns it on:

1. Open Hand Control and go to the Agents tab.
2. Tick Allow local programs to connect (agent API). The line under it shows On - 127.0.0.1:47294 (this PC only).
3. Optional, off by default:
   • Let agents turn control on/off and change the program enables POST /v1/control and POST /v1/profile.
   • Let agents take a camera snapshot (shows you) enables GET /v1/camera.jpg.

The API starts with the app from then on, until the box is unticked. If it cannot start, for example because another program already uses the port, Hand Control shows an error and unticks the box itself.

Connecting

Signed requests (recommended): the key never leaves your program

When Hand Control is closed, any other program could open port 47294. A Bearer header would hand that program the key. With a signed request it only sees a one-off signature, and every reply to an accepted signed request is signed back, so you also know the answer came from Hand Control, on the same connection. This is what the MCP connector does.

Authorization: HC-HMAC-SHA256 ts=<unix seconds>, nonce=<32 lowercase hex>, sig=<64 lowercase hex>

sig = hex HMAC-SHA256(key, the six lines below joined with "\n" (no newline at the end))
      hand-control-request
      METHOD                        GET or POST
      TARGET                        path + query, exactly as sent
      ts                            the same number as in the header
      nonce                         the same hex as in the header
      hex SHA-256 of the body       of the empty string for GET

• TARGET is the path with its query string exactly as sent (for example /v1/events?after=3&wait=25).
• ts must be within 60 s of the PC’s clock.
• Each nonce works once (use 16 random bytes).
• The reply carries X-HandControl-Signature = hex HMAC-SHA256(key, these four lines joined with \n: hand-control-response, the nonce, the HTTP status, hex SHA-256 of the reply body). If it is missing or wrong, do not trust the reply. A refusal made before the signature is checked is not signed: a wrong Host, a browser, a chunked body or a missing Content-Length (411), a body shorter than its Content-Length (400) or larger than 4 KB (413), a bad, stale or replayed signature (such as one made with a wrong key), the 503 sent when more than 8 connections are open, and the plain HTML replies below. A GET sent with no HTTP version is not signed either, even when it is accepted: its reply has no headers at all (see Endpoints).

Bearer requests get no reply signature. If you use Bearer, you can at least check the server first with GET /v1/hello?challenge=<32-128 lowercase hex>, the one request that needs no key. It returns proof = hex HMAC-SHA256(key, "hand-control-hello:" + challenge). This does not protect against the port changing hands between that check and your request; signed requests do.

The endpoints, their replies and two worked examples are in the Endpoints tab.

Back to the contents

Endpoint reference

Open an endpoint to see its reply and fields. Every request except GET /v1/hello needs the key, sent as described in Connecting.

How errors come back:

• JSON: every error has the shape {"error": {"code": "...", "message": "..."}}, except the three below.
• Too many connections: with more than 8 open, the 503 has an empty body and Retry-After: 1.
• Unknown method: a method other than GET, POST, PUT, DELETE and OPTIONS gets a plain HTML 501.
• Unreadable request: a plain HTML 400 (414 for a request line over 64 KB, 431 for 100 or more header lines or a header line over 64 KB).
• No status line or headers: when the request line has no HTTP version, a malformed one, or HTTP/2.0 or later, the HTML page is sent alone, so only its text names the error: 400, or 505 for HTTP/2.0 or later. The exception is a GET with no HTTP version (GET /v1/status): it is still answered, as HTTP/0.9, so its body arrives alone, with no reply signature.

{
  "app": "Hand Control", "version": "1.2.5", "api": 1,
  "camera_running": true, "camera_connected": true, "fps": 30.0,
  "control": true, "gesture": "point",
  "hands": ["Right"], "hands_seen": 1,
  "only_me": true, "owner": "recognised",
  "profile": "solidworks",
  "allowed": {"control": false, "camera": false},
  "events": {"stream": "9f2c41ab", "latest": 118}
}

{
  "time": 1790446131.371, "age_s": 0.03, "frame": 5120,
  "hands": [
    {"label": "Right",
     "landmarks": [[0.512, 0.604], [0.498, 0.571], "... 21 points ..."],
     "fingers_up": ["index"]}
  ]
}

{"stream": "9f2c41ab", "restarted": false, "next": 121, "latest": 121, "more": false, "missed": false, "events": [
  {"id": 119, "time": 1790446131.9, "type": "button", "button": "M", "action": "down", "pos": [812, 440]},
  {"id": 120, "time": 1790446131.9, "type": "gesture", "gesture": "rotate", "previous": "two hands"},
  {"id": 121, "time": 1790446131.9, "type": "two_hand_start", "mode": "rotate"}
]}

{"current": "solidworks", "profiles": [
  {"id": "solidworks", "name": "SolidWorks", "rotate": "middle-drag", "pan": "Ctrl+middle-drag", "zoom": "wheel"},
  {"id": "fusion360", "name": "Fusion 360", "rotate": "Shift+middle-drag", "pan": "middle-drag", "zoom": "wheel"}
]}

{"enabled": true}   ->   {"control": true}

{"profile": "fusion360"}   ->   {"profile": "fusion360"}

{"app": "Hand Control", "version": "1.2.5", "api": 1, "proof": "<64 lowercase hex>"}

import hashlib, hmac, json, os, secrets, time, urllib.error, urllib.request

KEY = open(os.path.expandvars(r"%APPDATA%\Hand Control\api-key.txt")).read().strip()
BASE = "http://127.0.0.1:47294"
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))   # never through a proxy

def mac(*parts):
    return hmac.new(KEY.encode(), "\n".join(parts).encode(), hashlib.sha256).hexdigest()

def call(method, target, body=None):
    data = json.dumps(body).encode() if body is not None else b""
    ts, nonce = str(int(time.time())), secrets.token_hex(16)
    sig = mac("hand-control-request", method, target, ts, nonce, hashlib.sha256(data).hexdigest())
    headers = {"Authorization": f"HC-HMAC-SHA256 ts={ts}, nonce={nonce}, sig={sig}"}
    if body is not None:
        headers["Content-Type"] = "application/json"
    req = urllib.request.Request(BASE + target, data=data if body is not None else None,
                                 method=method, headers=headers)
    try:
        with opener.open(req, timeout=40) as r:
            status, got, raw = r.status, r.headers.get("X-HandControl-Signature", ""), r.read()
    except urllib.error.HTTPError as e:
        status, got, raw = e.code, e.headers.get("X-HandControl-Signature", ""), e.read()
    if status == 503 and not raw:
        raise RuntimeError("Hand Control is busy (more than 8 connections); retry in a second")
    if not got and status >= 400:
        # refused before the signature was checked (clock off, nonce reused, wrong key, ...): not signed,
        # so the reason it gives cannot be verified
        try:
            err = json.loads(raw)["error"]
            reason = f'{err["code"]}: {err["message"]}'
        except (ValueError, KeyError, TypeError):
            reason = "no JSON error"
        raise RuntimeError(f"refused before signing (HTTP {status}); unverified reason: {reason}")
    want = mac("hand-control-response", nonce, str(status), hashlib.sha256(raw).hexdigest())
    if not hmac.compare_digest(got, want):
        raise RuntimeError("not Hand Control (or the key was replaced)")
    if status >= 400:
        raise RuntimeError(json.loads(raw)["error"]["message"])
    return json.loads(raw)

ev = call("GET", "/v1/status")["events"]
after, stream = ev["latest"], ev["stream"]
while True:
    r = call("GET", f"/v1/events?after={after}&wait=25&stream={stream}")
    after, stream = r["next"], r["stream"]
    for e in r["events"]:
        if e["type"] == "gesture":
            print(e["previous"], "->", e["gesture"])

$key = Get-Content "$env:APPDATA\Hand Control\api-key.txt"
Invoke-RestMethod http://127.0.0.1:47294/v1/status -Headers @{ Authorization = "Bearer $key" }

Versioning

Every path starts with /v1. New fields may be added to replies; clients should ignore fields they do not know. A breaking change will get a new version prefix.

Back to the contents