plesty.lib.experiment.base_experiment ===================================== .. py:module:: plesty.lib.experiment.base_experiment .. autoapi-nested-parse:: Asynchronous Experiment ABC with plan execution and checkpoint/resume. An :class:`Experiment` orchestrates one or more Devices to execute a measurement routine (issue plesty-lib#3). Its public lifecycle is ``setup()`` → ``run()`` → ``teardown()``, all asynchronous; ``run()`` is a template method that executes the experiment's :class:`~.schedule.Plan` of atomic steps with crash-safe status logging and resume — analogous to checkpointing in deep learning: the plan is the training schedule, the journal is the training log, and the completed-step set plus persisted step results form the checkpoint. Every run owns a directory (the convention is spelled out once, in :mod:`plesty.lib.experiment.runs`, which also reads it back): .. code-block:: text // # run_id = _ ├── plan.json # frozen schedule + config + content hash ├── journal.jsonl # append-only event log (status, resume source) ├── records.jsonl # one line per completed step: its result document └── data/ # blobs the records reference (arrays, images) └── step_0003.npy Blobs written by devices themselves (frames, images) never cross the network: they go to a shared disk under ``//raw``, in the acquiring host's spelling of the share (:attr:`Experiment.raw_dir`). The share is named once, by environment — ``PLESTY_DATA_DIR`` as the host sees it, ``PLESTY_DATA_MOUNT`` as this machine sees it — and both spellings are journaled with ``run_started`` so any machine can read the run back. With the mount set, ``run_root`` defaults to it: runs land on the share too. Minimal runnable example (mock, no hardware): .. code-block:: python import asyncio from plesty.lib.data import PlestyArray from plesty.lib.experiment import Experiment, Plan, Step class LineScan(Experiment): def build_plan(self) -> Plan: steps = [ Step(id=f"scan[x={x}]", op="scan_point", params={"x": x}) for x in range(5) ] return Plan(steps, config={"points": 5}) def scan_point(self, x: int) -> PlestyArray: return PlestyArray([float(x)], name="signal", unit="a.u.") run_id = asyncio.run(LineScan(run_root="runs").run()) # ... interrupted? Resume, skipping completed steps: asyncio.run(LineScan(run_root="runs").run(resume=run_id)) Attributes ---------- .. autoapisummary:: plesty.lib.experiment.base_experiment._RESERVED_OPS Exceptions ---------- .. autoapisummary:: plesty.lib.experiment.base_experiment.PlanMismatchError plesty.lib.experiment.base_experiment.InvalidOperationError Classes ------- .. autoapisummary:: plesty.lib.experiment.base_experiment.Experiment Module Contents --------------- .. py:exception:: PlanMismatchError Bases: :py:obj:`RuntimeError` Raised when resuming a run whose stored plan differs from the current one. Initialize self. See help(type(self)) for accurate signature. .. py:exception:: InvalidOperationError Bases: :py:obj:`RuntimeError` Raised when a plan step references an invalid experiment operation. Initialize self. See help(type(self)) for accurate signature. .. py:data:: _RESERVED_OPS .. py:class:: Experiment(devices: Optional[plesty.lib.device.composite_device.CompositeDevice] = None, run_root: str | pathlib.Path | None = None, name: Optional[str] = None, max_retries: int = 3, retry_sleep: float = 3.0, data_dir: Optional[str] = None, data_mount: Optional[str] = None) Bases: :py:obj:`abc.ABC` Base class for PLESTY experiments with atomic scheduling and resume. Subclasses override exactly three lifecycle hooks: * :meth:`build_plan` — **required**: the atomic measurement schedule. * :meth:`setup` / :meth:`teardown` — optional: extend device connection/release (call ``await super().setup()`` / ``...teardown()``). plus one public method per step operation. Step operations are validated before a run starts — every ``Step.op`` must name an existing public callable on the experiment and must not be one of the reserved lifecycle methods above. Device access goes through the :class:`~plesty.lib.device.composite_device.CompositeDevice` passed at construction — an Experiment receives Device instances, never raw hardware. Initialize the experiment. :param devices: Composite device holding all instruments this experiment orchestrates; connected in :meth:`setup` and disconnected in :meth:`teardown`. :param run_root: Directory under which each run creates its own checkpoint directory; ``None`` takes ``PLESTY_DATA_MOUNT`` from the environment (runs land on the share) and falls back to ``runs``. :param name: Experiment name used in run ids; defaults to the subclass name in lowercase. :param max_retries: Attempts per step before the run aborts. :param retry_sleep: Seconds to wait between step retries. :param data_dir: The shared data root as the acquiring host sees it; ``None`` takes ``PLESTY_DATA_DIR``. Device blobs go to :attr:`raw_dir` beneath it. :param data_mount: The same root as this machine sees it; ``None`` takes ``PLESTY_DATA_MOUNT``. Journaled for readers. .. py:attribute:: devices :value: None .. py:attribute:: data_dir .. py:attribute:: data_mount .. py:attribute:: run_root .. py:attribute:: name :value: '' .. py:attribute:: max_retries :value: 3 .. py:attribute:: retry_sleep :value: 3.0 .. py:attribute:: run_id :type: Optional[str] :value: None .. py:attribute:: run_dir :type: Optional[pathlib.Path] :value: None .. py:attribute:: logger .. py:attribute:: _plan_length :value: 0 .. py:property:: raw_dir :type: Optional[str] Where this run's device-written blobs go, in the host's spelling. ``//raw`` — one folder per run, so a directory the device lists on every acquisition never grows across runs, and a resumed run keeps writing where it started. ``None`` without a data root or before a run has started. .. py:method:: build_plan() -> plesty.lib.experiment.schedule.Plan :abstractmethod: Build the atomic measurement schedule for this experiment. Must be deterministic for a given configuration: resume relies on the regenerated plan matching the persisted one by content hash. .. py:method:: setup() -> None :async: Prepare the experiment; the default connects and preflights all devices. :raises RigNotReadyError: If a sub-device does not answer or is not the device its configuration requires (see :meth:`CompositeDevice.preflight`). Nothing is measured on a rig that is not what the plan assumes. .. py:method:: teardown() -> None :async: Release resources; the default disconnects all devices. Always awaited by :meth:`run`, whether the run completes, fails, or is canceled. .. py:method:: run(resume: Optional[str] = None) -> str :async: Execute the plan, journaling every step; optionally resume a run. :param resume: Run id of an earlier stopped/canceled/aborted run to continue. Steps journaled as completed are skipped; the stored plan must match the current one by content hash. :returns: The run id (directory name under ``run_root``) — pass it back as *resume* to continue after an interruption. :raises InvalidOperationError: If a plan step references a missing, private, or reserved operation (checked before anything runs). :raises PlanMismatchError: If resuming and the stored plan differs from the plan the experiment generates now. :raises Exception: The final step error is re-raised after ``run_aborted`` is journaled and :meth:`teardown` has completed. .. py:method:: _validate_plan(plan: plesty.lib.experiment.schedule.Plan) -> None Check that every step operation is a valid public method. Runs before any device is touched or any journal written, so a misconfigured plan fails fast instead of aborting mid-measurement. :raises InvalidOperationError: Listing every offending step. .. py:method:: _prepare_run(plan: plesty.lib.experiment.schedule.Plan, resume: Optional[str]) -> tuple[str, set[str]] Create a new run directory or validate and load a resumed one. :returns: The run id and the set of step ids already completed — those the journal marks completed plus those with a committed record. .. py:method:: _run_step(step: plesty.lib.experiment.schedule.Step, index: int, run_dir: pathlib.Path, journal: plesty.lib.experiment.journal.Journal) -> None :async: Execute one atomic step with retries and persist its result.