# Device Test Helpers

Smaller tools for testing parts of a device while you write it, before the full
[Testing a Device](device.md) pipeline, or for things it does not cover. All are in
`plesty.lib.test`. None of them imports pytest: a failure raises `AssertionError`,
which fails a pytest test and can equally be run from a plain script next to the
instrument.

| Helper | Tests |
|---|---|
| `device_param_system.auto_test` | reading and writing every parameter |
| `device_func_system.auto_test` | calling every operation |
| `grouped_param_keys` | how parameters in groups are named |
| `resource_allocation` | several clients sharing one instrument |
| `schema_params` | refreshing a parameter file from a real instrument |

## Every parameter: `device_param_system.auto_test`

Opens the device, prints its identity, and then goes through every parameter: a
read-only one is read; a read-write one is read and the value written back; a
write-only one is skipped. It prints which passed and which failed, and raises
`AssertionError` if any failed.

```python
from plesty.lib.test.device_param_system import auto_test

from my_device import MyDevice

auto_test(MyDevice, "mock", use_mock_solver=True, seed=42)        # simulated instrument
auto_test(MyDevice, "USB0::0x1313::0x8078::P0000001::INSTR",      # the real one
          use_mock_solver=False, ignore_keys=["DANGEROUS_PARAM"])
```

| Argument | Default | Meaning |
|---|---|---|
| arguments after the class | | passed to its constructor, typically the address |
| `use_mock_solver` | `True` | answer from a simulated instrument that keeps written values in memory, instead of talking to hardware |
| `ignore_keys` | `None` | parameters to skip |
| `seed` | `123` | makes the generated values repeatable |
| `sleep_time` | `0.1` | seconds to wait between parameters on a real instrument; ignored with the simulated one |
| `device_kwargs` | `None` | keyword arguments for the constructor |

## Every operation: `device_func_system.auto_test`

The same for operations: each is called with arguments generated from its declared
inputs (optional ones left out at random), and must return a dictionary.

```python
from plesty.lib.test.device_func_system import auto_test

auto_test(MyDevice, "mock", use_mock_solver=True)
auto_test(MyDevice, "USB0::…::INSTR", use_mock_solver=False, ignore_ops=["reset", "calibrate"])
```

It takes the same arguments, with `ignore_ops` in place of `ignore_keys`. On a real
instrument, list every operation that moves, emits or resets in `ignore_ops`.

## Parameters in groups: `grouped_param_keys`

A parameter in a group is named `GROUP.key`, such as `MOTOR.position`; one in the
default group is named by its key alone. Three ready-made pytest tests check a device
follows that rule. Import them into a test file and pytest runs them:

```python
from plesty.lib.test.grouped_param_keys import (
    test_default_param_uses_bare_key_only,
    test_get_config_list_returns_full_keys_for_grouped_params,
    test_grouped_param_requires_full_key,
)
```

## Several clients: `resource_allocation`

A server can let several clients share one instrument, each claiming the parts it
uses (see [Multi-Client Access Control](../device/service.md#multi-client-access-control)).
`assert_resource_manager_client_allocation` starts a server process and one process
per client, lets the clients claim their parts in phases, and checks who got what:

```python
from plesty.lib.test.resource_allocation import (
    ClientProcessConfig,
    assert_resource_manager_client_allocation,
)


def test_two_clients_cannot_claim_the_same_line():
    assert_resource_manager_client_allocation(
        device_cls=MyDevice,
        device_args={"port": 5555},
        client_configs=[
            ClientProcessConfig(client_id="client1", resources=["Dev1/port0/line0"], phase=0,
                                hold_seconds=10,
                                expected_connection_info={"client_id": "client1",
                                                          "resources": ["Dev1/port0/line0"]}),
            ClientProcessConfig(client_id="client2", resources=["Dev1/port0/line0"], phase=1,
                                expected_connection_info={}),          # refused: already taken
            ClientProcessConfig(client_id="client3", resources=["Dev1/port0/line1"], phase=1,
                                expected_connection_info={"client_id": "client3",
                                                          "resources": ["Dev1/port0/line1"]}),
        ],
    )
```

| `ClientProcessConfig` field | Meaning |
|---|---|
| `client_id` | the client's name |
| `resources` | the parts it claims when it connects |
| `phase` | when it runs: lower phases start first |
| `hold_seconds` | how long it keeps its claim after connecting |
| `expected_connection_info` | what the client must report after connecting; empty means the claim must be refused |

## Refreshing a parameter file: `schema_params`

`test_real_device_refreshes_schema_defaults_and_types` connects to a real instrument,
reads every parameter, and writes a copy of the module's parameter file with each
type and default taken from what the instrument returned, as
`<name>_refreshed.json` next to the original. Parameters that could not be read are
left out of the copy, and a warning names them. Review the copy before replacing the
original.

```python
from plesty.lib.test.schema_params import test_real_device_refreshes_schema_defaults_and_types

test_real_device_refreshes_schema_defaults_and_types(
    MyDevice,
    param_schema="config/my_device_params.json",
    address="USB0::0x1313::0x8078::P0000001::INSTR",
)
```

It is not collected by pytest automatically. The field test offers the same with a
review step: see `propose_schema_update()` in
[Testing on the Instrument](field_test.md).

## A suggested order

1. After declaring the parameters, run `device_param_system.auto_test` with the
   simulated instrument.
2. After declaring the operations, run `device_func_system.auto_test` the same way.
3. Run both against the real instrument, with anything dangerous in `ignore_keys` and
   `ignore_ops`.
4. Run the full [Testing a Device](device.md) pipeline, and then
   [Testing on the Instrument](field_test.md).