Experiment Framework

An experiment runs a measurement: a list of steps that use devices, with every result saved as it arrives. plesty-lib’s experiment framework, in plesty.lib.experiment, takes care of everything around your steps: saving each result the moment it exists, surviving a crash, continuing where a run stopped, checking the devices first, and starting the whole thing from a command line.

If you have not written an experiment yet, start with An Experiment in the quick start. This chapter explains each part in full.

The parts

Part	What it is
Step	one unit of work: an id, the name of the method to call (op), and the values to call it with (params). A step either finishes and is saved, or is run again when the run continues.
Plan	the full list of steps, plus the settings it was built from. Saved when the run starts and never changed afterwards; its fingerprint (content_hash()) identifies it.
Experiment	the base class you inherit. It checks the plan, runs the steps, retries failed ones, saves results, and continues stopped runs.
the journal	a log of everything that happened in a run, one line per event, only ever added to. Reading it back tells where a run got to.
the records	one line per finished step, holding what the step returned
Run	reads a finished or running run back: its plan, settings, state and records
ExperimentCLI	the standard command line that starts an experiment

Each run keeps all of this in its own folder. The run folder lists what the folder holds and who writes each file.

In this chapter

Page	Covers
Writing an Experiment	the plan, step methods, retries, setting up and tearing down devices, reporting progress
The Command Line	ExperimentCLI: its options, the functions you register with it, and what it declares about your experiment
Checking the Devices Before a Run	making sure every device answers and is the device you expect, and continuing without an optional one
The Setup File	setup.yaml: which computers the devices run on
Data on a Shared Disk	letting devices on other computers save large files into the run
Stopping and Continuing a Run	crashes, Ctrl-C, requests to pause or stop, and continuing a run
Reading a Run	finding runs and reading their plan, state and records
Testing an Experiment	the tests an experiment module runs on itself, without instruments