Remote benches: the agent and `plesty.bench`

Nothing on an experiment PC needs the bench application: experiments talk to the device servers directly. The one remote user is the developer running the install → update → field-test loop from their own machine. For that, the bench runs an agent and the developer installs plesty-bench — a client of a few hundred lines, and the only thing a developer machine needs. The repository publishes two distributions to PyPI: this client, and plesty-server for the bench itself.

Two deliverables, one repository

	`plesty.bench`	`plesty.server`
What	The client: `BenchClient` + the `plesty-bench` command	The bench application: model, services, presenter, CLI, console, agent
Ships as	wheel on PyPI, distribution `plesty-bench`	wheel on PyPI, distribution `plesty-server` (assembled by `packaging/server_dist.py`); light installers run `uv tool install "plesty-server[gui]"`; a source checkout (`uv sync`) for development
Depends on	`pyzmq`, `click`	`plesty-lib`, `pyyaml`, `pyzmq`, `click` (+ `plesty-lib[gui]` for the console)
Imports	nothing from `plesty.server`	`plesty.bench` (the console’s remote mode)

pyproject.toml makes the split: only-include = ["plesty/bench"] for this project’s wheel, plesty-bench as its only console script; the server’s dependencies are the server extra (and the dev group, so uv sync in a checkout has everything). The version and the CHANGELOG are shared, and describe reports the server version so a client can tell.

The contract: the CLI is the API

The agent accepts a plesty-server argument list and returns the exit code and the output. Anything the command line can do is remote, with no second schema to keep in step. Three request types on the plesty-lib device-server wire (ZMQ DEALER → ROUTER, JSON, status: ok|error):

{"type": "describe"}
  → {"status": "ok", "result": {"kind": "plesty-server-agent", "version": "0.1.0",
                                 "home": "…", "fleet": "…", "commands": ["declare", …]}}
{"type": "exec", "argv": ["status", "--json"], "token": "…"}
  → {"status": "ok", "result": {"code": 0, "stdout": "[…]", "stderr": ""}}
{"type": "fetch", "path": "repos/plesty-pm100d/exp/reports/pm100d-s120c/field-test.json", "token": "…"}
  → {"status": "ok", "result": {"path": "…", "size": 2210, "data": "<base64>"}}

Because it is the device-server wire, plesty.lib.service.build_client and any describe probe understand the agent without plesty-bench.

Long work fits without streaming: install/update/field-test --no-wait return a job id, and jobs log <id> --offset N --json returns {text, offset, running, returncode, state} — BenchClient.follow() polls that until the job ends. list, status, reports, jobs list take --json for structured results.

Bench side: `plesty-server agent`

export PLESTY_BENCH_TOKEN=…          # or --token; empty = no check (trusted lab network)
plesty-server agent                  # tcp://*:5550, the home/fleet of this invocation

Every exec runs the click application in-process with captured output and the agent’s --home/-f prepended — a fresh Bench per command, so the agent and a local CLI see the same files.
Requests run on a worker pool; a describe is answered while a start waits for its probe. Replies go back through the socket thread.
exec refuses agent and gui; fetch is confined to the bench home and capped at 16 MB; the token is compared in constant time. Same trust model as the device servers: the lab network, no TLS.

Developer side: `plesty-bench`

uv tool install plesty-bench         # or: uvx plesty-bench …
export PLESTY_BENCH=lab-bench-03 PLESTY_BENCH_TOKEN=…

plesty-bench ping
plesty-bench status
plesty-bench update pm100d-dev --ref feat/drain        # started --no-wait on the bench, followed here
plesty-bench field-test pm100d-dev --gates connect,drain
plesty-bench reports pm100d-dev --json
plesty-bench fetch repos/plesty-pm100d/feat-drain/reports/pm100d-s120c/field-test.json -o .
plesty-bench -- start pm100d       # `--` when the remote arguments start with a dash

From Python:

from plesty.bench import BenchClient

with BenchClient("lab-bench-03", token="…") as bench:
    job = bench.exec(["field-test", "pm100d-dev", "--no-wait"]).stdout.strip()
    code = bench.follow(job, print)
    report = bench.fetch(
        bench.exec(["reports", "pm100d-dev", "--json"]).json()[0]["home_relative"]
    )

Without an agent

ssh lab-bench-03 plesty-server status needs neither the agent nor plesty-bench — only OpenSSH on the bench (available on Windows too). It is the fallback where a second listening port is unwelcome.

Remote benches: the agent and plesty.bench