plesty.lib.experiment.runs

How a run is named, laid out, found, and read back.

Experiment writes runs; this module is the other side — the one place that spells out the run-directory convention so viewers, analyzers, and command lines never re-derive it:

<run_root>/<name>_<YYYYmmdd-HHMMSS>/     # run id = experiment name + start time
├── plan.json                              # frozen schedule + config
├── journal.jsonl                          # events; run_started carries the data paths
├── records.jsonl                          # one line per completed step: its result
└── data/step_NNNN.npy                     # blobs the records reference, if any

Device-written blobs (spectrometer frames, camera images) never cross the network: they land on a shared data disk that every bench mounts, each under its own path — its shared point. Each process is told where the disk is on its own machine — PLESTY_DATA — and every path that crosses between machines is relative to it. An experiment’s runs go under its root there (pol-pl/<run_id>), a run routes each device to its raw/<device> folder, and records a file as pol-pl/<run_id>/raw/spec/frame.spe, which a reader anywhere joins to its own PLESTY_DATA.

run = Run(latest_run("/mnt/share", name="pol_pl"))
run.config["powermeter"]          # the frozen configuration
run.status()["state"]             # "running" / "completed" / ...
run.local(record["data_file"])    # shared-relative path → path on this machine

Attributes

`BLOB_SUBDIR`
`RECORDS_FILE`
`DATA_ROOT_VAR`
`PLAN_FILE`
`JOURNAL_FILE`
`PROGRESS_FILE`
`RAW_SUBDIR`
`RUN_ID_TIME_FORMAT`

Classes

Run

Read access to one run directory: plan, journal, records, data paths.

Functions

`run_id`(→ str)	Return the run id for experiment name started at when (now by default).
`parse_run_id`(→ Optional[tuple[str, datetime.datetime]])	Split a run id into its experiment name and start time.
`list_runs`(→ list[pathlib.Path])	Return the run directories under run_root, oldest first.
`latest_run`(→ pathlib.Path)	Return the most recently started run under run_root.
`data_root`(→ Optional[str])	The shared data disk as this machine sees it: `PLESTY_DATA`.
`run_root_under`(→ pathlib.Path)	Where an experiment's runs go: run_root, under the shared point root when relative.
`is_absolute`(→ bool)	Whether path is absolute on any system — `/mnt/x`, `G:\x` or `\\server\x`.

Module Contents

plesty.lib.experiment.runs.BLOB_SUBDIR = 'data'

plesty.lib.experiment.runs.RECORDS_FILE = 'records.jsonl'

plesty.lib.experiment.runs.DATA_ROOT_VAR = 'PLESTY_DATA'

plesty.lib.experiment.runs.PLAN_FILE = 'plan.json'

plesty.lib.experiment.runs.JOURNAL_FILE = 'journal.jsonl'

plesty.lib.experiment.runs.PROGRESS_FILE = 'progress.json'

plesty.lib.experiment.runs.RAW_SUBDIR = 'raw'

plesty.lib.experiment.runs.RUN_ID_TIME_FORMAT = '%Y%m%d-%H%M%S'

plesty.lib.experiment.runs.run_id(name: str, when: datetime.datetime | None = None) → str

Return the run id for experiment name started at when (now by default).

Parameters:

name (str)
when (Optional[datetime.datetime])

Return type:

str

plesty.lib.experiment.runs.parse_run_id(text: str) → tuple[str, datetime.datetime] | None

Split a run id into its experiment name and start time.

Parameters:: text (str) – A directory name.
Returns:: (name, started), or None when text is not a run id — the name may itself contain underscores, so the split is at the last one.
Return type:: Optional[tuple[str, datetime.datetime]]

plesty.lib.experiment.runs.list_runs(run_root: str | pathlib.Path, name: str | Sequence[str] | None = None) → list[pathlib.Path]

Return the run directories under run_root, oldest first.

Parameters:

run_root (str | pathlib.Path) – The directory runs are created in.
name (Union[str, Sequence[str], None]) – Keep only runs of this experiment name (or any of several — an experiment that was renamed still owns its old runs). None keeps every run.

Return type:

list[pathlib.Path]

plesty.lib.experiment.runs.latest_run(run_root: str | pathlib.Path, name: str | Sequence[str] | None = None) → pathlib.Path

Return the most recently started run under run_root.

Raises:

FileNotFoundError – If there is no such run yet — a viewer opened before its experiment is a mistake worth naming.

Parameters:

run_root (str | pathlib.Path)
name (Union[str, Sequence[str], None])

Return type:

pathlib.Path

plesty.lib.experiment.runs.data_root(env: collections.abc.Mapping[str, Any] | plesty.lib.utils.settings.EnvSettings | None = None) → str | None

The shared data disk as this machine sees it: PLESTY_DATA.

A leading ~ is this machine’s home, so ~/plesty never becomes a folder named ~ under the working directory (#69).

Parameters:: env (Union[collections.abc.Mapping[str, Any], plesty.lib.utils.settings.EnvSettings, None]) – Where to read; the process environment and .env by default.
Return type:: Optional[str]

plesty.lib.experiment.runs.run_root_under(run_root: str | pathlib.Path | None, root: str | None) → pathlib.Path

Where an experiment’s runs go: run_root, under the shared point root when relative.

An experiment names its root relative to the shared data disk — pol-pl — so its runs land in the same folder whichever bench looks: /mnt/plesty/pol-pl here, G:\Plesty\pol-pl there. An absolute one is taken as it is; None is the shared point itself, and runs without one. A leading ~ on either is this machine’s home, so ~/runs is absolute and is never joined under the share (#69).

Parameters:

run_root (str | pathlib.Path | None) – The experiment’s root, as given.
root (Optional[str]) – This machine’s view of the shared disk (PLESTY_DATA).

Return type:

pathlib.Path

plesty.lib.experiment.runs.is_absolute(path: str) → bool

Whether path is absolute on any system — /mnt/x, G:\x or \\server\x.

A record may have been written on another system than the one reading it, so this machine’s own rule is not the one to ask.

Parameters:: path (str)
Return type:: bool

class plesty.lib.experiment.runs.Run(run_dir: str | pathlib.Path, *, data_root: str | None = None, replay_rows: int = 0)

Read access to one run directory: plan, journal, records, data paths.

Nothing here writes. The plan is read once; the journal is replayed on each status() because a run under way keeps appending to it.

Attach to a run directory.

Parameters:

run_dir (str | pathlib.Path) – The run directory.
data_root (Optional[str]) – This machine’s view of the shared data disk; None takes PLESTY_DATA from the environment, then the root journaled by the machine that ran the experiment (only when that directory exists here).
replay_rows (int) – How source() hands records out — 0 follows the run live, n rations a stored run n records per poll.

path

replay_rows = 0

_data_root = None

_plan: plesty.lib.experiment.schedule.Plan | None = None

_started: dict[str, Any] | None = None

sources: list[Any] = []

property id: str

The run id (the directory name).

Return type:: str

property name: str | None

The experiment name the run id carries, if it parses as one.

Return type:: Optional[str]

property started: datetime.datetime | None

The start time the run id carries, if it parses as one.

Return type:: Optional[datetime.datetime]

property plan: plesty.lib.experiment.schedule.Plan

The frozen plan (loaded once).

Return type:: plesty.lib.experiment.schedule.Plan

property config: dict[str, Any]

The frozen configuration; {} when the plan is unreadable.

Return type:: dict[str, Any]

property journal: plesty.lib.experiment.journal.Journal

The run’s journal.

Return type:: plesty.lib.experiment.journal.Journal

status() → dict[str, Any]

The run status by journal replay (see Journal.status()).

Return type:: dict[str, Any]

progress() → dict[str, Any]

How far the run has got, and how much longer it is likely to take.

Everything here is derived from what the run already wrote — the plan’s length, the journal’s step timings, and whatever a step in flight last reported — so watching costs a directory read and never disturbs the measurement.

The estimate is a mean over completed steps, which is honest for a sweep of like steps and wrong for a plan whose steps differ in kind; it is offered as None until at least one step has completed rather than as a number derived from nothing.

Returns:: A mapping with state, completed and failed step counts, total steps in the plan, fraction of the plan done, elapsed_s since the run started, step_s (the mean completed step), eta_s (None while unknowable), step (what the step in flight last reported, or None) and reports.
Return type:: dict[str, Any]

step_progress() → dict[str, Any] | None

What the step in flight last said about itself, if anything.

A step is atomic and the journal speaks only at its boundaries, so a long exposure or a magnet ramp is indistinguishable from a hung run without this. A step that reports nothing simply has no entry.

Returns:: The reported progress — step_id, done, total, detail and the time it was written — or None.
Return type:: Optional[dict[str, Any]]

_elapsed() → float | None

Seconds between the run starting and its last event.

Returns:: The span, or None when the journal carries no usable times.
Return type:: Optional[float]

records() → Iterable[dict[str, Any]]

Iterate the committed step records, oldest first.

Return type:: Iterable[dict[str, Any]]

_run_started() → dict[str, Any]

Return type:: dict[str, Any]

property provenance: dict[str, Any]

What was running when the run started; {} for a run without it.

Runs written before this was journaled have none, and say so by being empty rather than by inventing today’s versions — which would be the one answer guaranteed wrong.

Return type:: dict[str, Any]

property raw_dir: str | None

<run_id>/raw, under each’s data path.

Type:: Where the run asked its devices to write
Return type:: Optional[str]

property data_root: str | None

The shared data disk as this machine sees it (see the class docstring).

Return type:: Optional[str]

local(path: str) → pathlib.Path

Return path, as a record spells it, as a path on this machine.

A path relative to the shared disk is joined to this machine’s view of it; an absolute one is taken as it is.

Parameters:: path (str)
Return type:: pathlib.Path

source(mapper: plesty.lib.monitor.sources.FrameMapper | None = None) → plesty.lib.monitor.sources.RunSource | plesty.lib.monitor.sources.ReplaySource

Return a source over this run’s records — live or replayed.

A RunSource when replay_rows is 0, else a ReplaySource handing out that many records per poll. Every source made here is kept in sources.

Make one source per consumer: a source hands each record out once, so two views polling the same source would each see half the rows. The mapper is what they share.

Parameters:: mapper (Optional[plesty.lib.monitor.sources.FrameMapper])
Return type:: Union[plesty.lib.monitor.sources.RunSource, plesty.lib.monitor.sources.ReplaySource]

property exhausted: bool

Whether every source made here has handed out its last record.

Live sources never are (the next record may still be coming); a run without sources counts as exhausted so a watcher does not wait forever.

Return type:: bool

__repr__() → str

Return Run('<path>').

Return type:: str