Device Test Helpers

Smaller tools for testing parts of a device while you write it, before the full Testing a Device pipeline, or for things it does not cover. All are in plesty.lib.test. None of them imports pytest: a failure raises AssertionError, which fails a pytest test and can equally be run from a plain script next to the instrument.

Helper	Tests
`device_param_system.auto_test`	reading and writing every parameter
`device_func_system.auto_test`	calling every operation
`grouped_param_keys`	how parameters in groups are named
`resource_allocation`	several clients sharing one instrument
`schema_params`	refreshing a parameter file from a real instrument

Every parameter: `device_param_system.auto_test`

Opens the device, prints its identity, and then goes through every parameter: a read-only one is read; a read-write one is read and the value written back; a write-only one is skipped. It prints which passed and which failed, and raises AssertionError if any failed.

from plesty.lib.test.device_param_system import auto_test

from my_device import MyDevice

auto_test(MyDevice, "mock", use_mock_solver=True, seed=42)        # simulated instrument
auto_test(MyDevice, "USB0::0x1313::0x8078::P0000001::INSTR",      # the real one
          use_mock_solver=False, ignore_keys=["DANGEROUS_PARAM"])

Argument	Default	Meaning
arguments after the class		passed to its constructor, typically the address
`use_mock_solver`	`True`	answer from a simulated instrument that keeps written values in memory, instead of talking to hardware
`ignore_keys`	`None`	parameters to skip
`seed`	`123`	makes the generated values repeatable
`sleep_time`	`0.1`	seconds to wait between parameters on a real instrument; ignored with the simulated one
`device_kwargs`	`None`	keyword arguments for the constructor

Every operation: `device_func_system.auto_test`

The same for operations: each is called with arguments generated from its declared inputs (optional ones left out at random), and must return a dictionary.

from plesty.lib.test.device_func_system import auto_test

auto_test(MyDevice, "mock", use_mock_solver=True)
auto_test(MyDevice, "USB0::…::INSTR", use_mock_solver=False, ignore_ops=["reset", "calibrate"])

It takes the same arguments, with ignore_ops in place of ignore_keys. On a real instrument, list every operation that moves, emits or resets in ignore_ops.

Parameters in groups: `grouped_param_keys`

A parameter in a group is named GROUP.key, such as MOTOR.position; one in the default group is named by its key alone. Three ready-made pytest tests check a device follows that rule. Import them into a test file and pytest runs them:

from plesty.lib.test.grouped_param_keys import (
    test_default_param_uses_bare_key_only,
    test_get_config_list_returns_full_keys_for_grouped_params,
    test_grouped_param_requires_full_key,
)

Several clients: `resource_allocation`

A server can let several clients share one instrument, each claiming the parts it uses (see Multi-Client Access Control). assert_resource_manager_client_allocation starts a server process and one process per client, lets the clients claim their parts in phases, and checks who got what:

from plesty.lib.test.resource_allocation import (
    ClientProcessConfig,
    assert_resource_manager_client_allocation,
)


def test_two_clients_cannot_claim_the_same_line():
    assert_resource_manager_client_allocation(
        device_cls=MyDevice,
        device_args={"port": 5555},
        client_configs=[
            ClientProcessConfig(client_id="client1", resources=["Dev1/port0/line0"], phase=0,
                                hold_seconds=10,
                                expected_connection_info={"client_id": "client1",
                                                          "resources": ["Dev1/port0/line0"]}),
            ClientProcessConfig(client_id="client2", resources=["Dev1/port0/line0"], phase=1,
                                expected_connection_info={}),          # refused: already taken
            ClientProcessConfig(client_id="client3", resources=["Dev1/port0/line1"], phase=1,
                                expected_connection_info={"client_id": "client3",
                                                          "resources": ["Dev1/port0/line1"]}),
        ],
    )

`ClientProcessConfig` field	Meaning
`client_id`	the client’s name
`resources`	the parts it claims when it connects
`phase`	when it runs: lower phases start first
`hold_seconds`	how long it keeps its claim after connecting
`expected_connection_info`	what the client must report after connecting; empty means the claim must be refused

Refreshing a parameter file: `schema_params`

test_real_device_refreshes_schema_defaults_and_types connects to a real instrument, reads every parameter, and writes a copy of the module’s parameter file with each type and default taken from what the instrument returned, as <name>_refreshed.json next to the original. Parameters that could not be read are left out of the copy, and a warning names them. Review the copy before replacing the original.

from plesty.lib.test.schema_params import test_real_device_refreshes_schema_defaults_and_types

test_real_device_refreshes_schema_defaults_and_types(
    MyDevice,
    param_schema="config/my_device_params.json",
    address="USB0::0x1313::0x8078::P0000001::INSTR",
)

It is not collected by pytest automatically. The field test offers the same with a review step: see propose_schema_update() in Testing on the Instrument.

A suggested order

After declaring the parameters, run device_param_system.auto_test with the simulated instrument.
After declaring the operations, run device_func_system.auto_test the same way.
Run both against the real instrument, with anything dangerous in ignore_keys and ignore_ops.
Run the full Testing a Device pipeline, and then Testing on the Instrument.

Device Test Helpers

Every parameter: device_param_system.auto_test

Every operation: device_func_system.auto_test

Parameters in groups: grouped_param_keys

Several clients: resource_allocation

Refreshing a parameter file: schema_params