Testing a Device

DevicePipeline, in plesty.lib.test.device_pipeline, is the standard test of a device module. It needs no instrument: it replaces the instrument with a simulated one that answers every parameter and operation with a made-up value of the declared type, and checks the device against it. It therefore runs anywhere, including in CI on every push.

The test file

A device module’s whole test file is:

# tests/test_mydevice.py
from plesty.lib.test.device_pipeline import DevicePipeline

from plesty.mydevice import Device

PIPELINE = DevicePipeline(Device, address="mock")


def test_mock_pipeline():
    PIPELINE.run_mock_pipeline()

Arguments after the class are passed to its constructor. plesty init mock-test, a command of plesty-sdk, PLESTY’s developer tool, writes this file for a module from what the module declares.

run_mock_pipeline() runs all eight tests below and reports every failure it finds, rather than stopping at the first. plesty-sdk looks for this test and runs it as gate d1 of plesty check before the module may be published (see Which tests each module runs).

What it checks

Test	Checks
`test_schema_integrity`	what the device declares (`doc_model()`) is consistent: every parameter has a usable type, its options match that type, no two parameters share a key, every operation output has a type
`test_param_key_resolution`	every key in `get_config_list()` can be looked up with `get_config(key)`
`test_params_mock`	every parameter can be read and written against the simulated instrument
`test_funcs_mock`	every operation accepts generated arguments
`test_lifecycle`	a `with` block connects and disconnects; the device reports itself ready (`is_operatable`) once connected
`test_identity`	`identity()` returns a non-empty string
`test_check_errors`	`check_errors()` returns an empty list for a healthy device
`test_state_coverage`	`state` includes every key of `get_config_list()`

The tests read the device itself, never its schema files. A parameter registered in Python code does not appear in the JSON file, and in a grouped schema the file’s keys are not the keys the running device answers to, so only the device is a reliable source.

What passing does not mean

These tests show that a module’s declarations are consistent and that its parameter and operation handling works. They cannot show that the instrument accepts the commands the module sends, that its driver is installed, or how the connection behaves when it goes wrong: a simulated instrument has none of these. A module that passes all eight can still fail on the instrument. That is what Testing on the Instrument is for.