# Field tests

A hub device module scaffolded with `plesty init field-test` carries its own
field test — the hardware-gated run that verifies the module against the
real instrument and publishes `reports/<instrument>/field-test.json`, the
artifact hub.plesty.net and manager.plesty.net read. `plesty-server` runs it
**on the bench, from the developer's machine**, without anyone walking over.

## Tiers

| Tier | Script (in the checkout) | Needs | Publishes |
|---|---|---|---|
| `host` | `tests/field_test.py` | the instrument itself — the device server must be **stopped** (unless `--address mock`) | `field-test.json` |
| `client` | `tests/field_test_client.py` | a **running** server (`tcp://…`) | `field-test-client.json` |

Both scripts exist only in a **git checkout** (see [Installing](installing.md))
and run with the checkout's interpreter. The markdown/JSONL beside the JSON
carry the address and the serial and stay on the bench.

## The pieces

- `FieldTestRun` (model) — the request: tier, address, gates, repetitions,
  `keep_going`, `keep_server`, extra args. `argv()` builds the script's
  arguments (the host tier gets `--report reports/field-test-run` for its
  internal pair); `command(python, checkout)` builds the full command line
  and fails with the "install from git and scaffold" hint when the script is
  missing; `needs_instrument` says whether the server has to be stopped.
- `JobRunner` (service) — runs that command as a job of kind `field-test`,
  in the checkout, with the instance's `env` from the fleet, so the address
  and the report archive come from the same place as for the server. The
  developer tails `jobs.log(id, offset)` and waits for the exit code.
- `FieldTestReport` (model) — reads what was published: ok, passed/failed/
  skipped, per-gate results, findings by severity. `find_reports(checkout,
  since=job.started_utc)` lists what *this* run produced.

## The choreography — `Bench.field_test_now`

```text
run = FieldTestRun(tier="host", address=None, gates=("connect", "drain"))
inst = manager.resolve(spec)                     # must be a git install
record = supervisor.record(name)
if run.needs_instrument and not run.keep_server and supervisor.alive(name):
    supervisor.stop(name)                        # the instrument is single-owner
if run.tier == "client" and not supervisor.alive(name):
    supervisor.start(spec, installer.python(inst))
job = jobs.start(name, JOB_KIND, run.command(installer.python(inst), checkout),
                 cwd=checkout, env=spec.env, detail=run.to_dict())
job = jobs.wait(job.id)                          # or stream jobs.store.log(...)
if record is not None:
    supervisor.relaunch(record)                  # back exactly as it was
reports = find_reports(checkout, since_utc=job.started_utc)
```

`relaunch(record)` is why `ProcessRecord` keeps the interpreter, module,
args, env and port: the server comes back with the same arguments and on the
same port a client connected to before the run.

From the command line:

```bash
plesty-server field-test pm100d --gates connect,drain        # host tier, stops/relaunches the server
plesty-server field-test pm100d --address mock               # simulator: server left alone
plesty-server field-test pm100d --tier client                # against the running server
plesty-server field-test pm100d --no-wait                    # returns the job id; `jobs log <id> --follow`
plesty-server reports pm100d                                 # what was published
```