Testing Over the Network

Testing on the Instrument drives the device directly, in the same program. An experiment never reaches an instrument that way: it goes through a client, the network and the device’s server (see Use it from another computer). A module can pass every check on the instrument and still fail the first time an experiment uses it, because the path in between was never tested. ClientFieldTest, in plesty.lib.test.client_field_test, runs its checks through that whole path.

Two ways to run it

Way	Created as	What it tells you
attached	`ClientFieldTest(address="tcp://192.168.1.40:5555")`	tests a server that is already running, the way the lab actually uses it, across the real network. Only these timings mean anything: some network problems need real network delays and never show on one computer. Checks that need to restart the server are skipped, since the test cannot restart a server on another computer.
spawned	`ClientFieldTest(DemoDevice)`	starts a server itself, in a separate process on this computer, and talks to it there. Needs no instrument and no second computer, so it can run in CI. Its timings describe this computer only.

The report says which way produced it.

# tests/field_test_client.py
from plesty.lib.test.client_field_test import ClientFieldTest

RUNNER = ClientFieldTest(
    address="tcp://192.168.1.40:5555",   # or a device class, to spawn a server
    params=["wavelength", "power"],      # a client cannot list the parameters itself
    safe_ops=["measure_power"],          # the same allow-list as on the instrument
    label="pm-over-lan",
)

report = RUNNER.run()

run() also writes the report when instrument, reports_dir and a project are known; see Reports.

What it checks

In this order:

Test	Checks
`test_connect`	a client connects to the server
`test_describe`	the server describes the device: its methods and declarations
`test_protocol_fidelity`	parameters keep their type across the network
`test_functions`	the allowed operations can be called over the network, in the way their kind says
`test_session_survives_a_bad_result`	a call whose result cannot be sent back fails alone, without ending the session for everyone
`test_timeout_recovery`	the client recovers after a timeout
`test_resource_allocation`	claiming parts of the instrument (`resources`) works over the network
`test_wrapper_modes`	the server’s two ways of running calls give the same results; spawned only, since it restarts the server

Arguments

Argument	Default	Meaning
`device_factory`	`None`	the first argument: a device class, or a function that makes a device, for a spawned server; left out when attaching
`address`	`None`	the server’s address; required when attaching, chosen automatically when spawning
`params`	`None`	the parameters to read and write over the network
`safe_ops`	`None`	the operations the test may call; nothing else is called
`op_kinds`	`None`	what each operation does, so that a motion is moved out and back (see Which operations may run)
`slow_op_payload`	`None`	how far to move a motion so it is still moving when a stop reaches it
`expected_types`	`None`	the type each parameter’s reply must have
`resources`	`None`	the parts of the instrument to claim in `test_resource_allocation`
`fixed_threading`	`True`	how a spawned server runs its calls; ignored when attaching
`baseline`	`None`	the instrument test’s published `field-test.json`, so both tests cover the same operations
`label`, `instrument`, `reports_dir`		as for Testing on the Instrument

adopt_baseline() takes the list of what to test from baseline. Reports include the computer’s name and the address: keep them within the lab.