Testing an Experiment

An experiment cannot be tried out properly without its instruments, but most of what makes it safe to run can be checked without them: that its plan is the same every time, that every step names a method, and that the plan survives being saved. ExperimentPipeline, in plesty.lib.test.experiment_pipeline, runs these checks. It never calls run(), so no device is needed. The words used here are introduced in Experiment Framework.

Test	Checks
`test_experiment_subclass`	the module has a public `Experiment` subclass that can be created without devices
`test_plan_deterministic`	`build_plan()` called twice gives the same fingerprint (`content_hash()`)
`test_plan_ops_resolve`	every step’s `op` names a method, checked the same way `run()` checks it
`test_plan_serializable`	saving and loading the plan (`Plan.save`, `Plan.load`) keeps its fingerprint
`test_lifecycle_hooks`	`setup()` and `teardown()`, if overridden, are `async`

An experiment module exposes one pytest test per check:

# tests/test_pipeline.py
from plesty.lib.test.experiment_pipeline import ExperimentPipeline

from plesty.power_scan import PowerScan

PIPELINE = ExperimentPipeline(PowerScan, {"wavelengths": [500.0, 600.0]})

def test_experiment_subclass():
    PIPELINE.test_experiment_subclass()

def test_plan_deterministic():
    PIPELINE.test_plan_deterministic()

def test_plan_ops_resolve():
    PIPELINE.test_plan_ops_resolve()

def test_plan_serializable():
    PIPELINE.test_plan_serializable()

def test_lifecycle_hooks():
    PIPELINE.test_lifecycle_hooks()

The arguments after the class are passed to its constructor, here the config that PowerScan takes. PIPELINE.run_mock_pipeline() runs all five checks at once.

plesty-sdk, PLESTY’s developer tool, looks for these five tests and runs them as gate e1 of plesty check before the module may be published (see Which tests each module runs).