plesty.lib.experiment.runs ========================== .. py:module:: plesty.lib.experiment.runs .. autoapi-nested-parse:: How a run is named, laid out, found, and read back. :class:`~plesty.lib.experiment.base_experiment.Experiment` writes runs; this module is the other side — the one place that spells out the run-directory convention so viewers, analyzers, and command lines never re-derive it: .. code-block:: text /_/ # run id = experiment name + start time ├── plan.json # frozen schedule + config ├── journal.jsonl # events; run_started carries the data paths ├── records.jsonl # one line per completed step: its result └── data/step_NNNN.npy # blobs the records reference, if any Device-written blobs (spectrometer frames, camera images) never cross the network: they land on a shared disk under ``//raw``, where ``data_dir`` is the share *as the acquiring host sees it*. A machine reading the run sees the same share under its own mount, so every path in a record is translated through the pair (``data_dir``, ``data_mount``). Both come from one environment convention — ``PLESTY_DATA_DIR`` and ``PLESTY_DATA_MOUNT`` — and the run journals the spelling it was written with, so a viewer on any machine needs at most its own mount. .. code-block:: python run = Run(latest_run("/mnt/share", name="pol_pl")) run.config["powermeter"] # the frozen configuration run.status()["state"] # "running" / "completed" / ... run.local(record["data_file"]) # host path → path on this machine Attributes ---------- .. autoapisummary:: plesty.lib.experiment.runs.BLOB_SUBDIR plesty.lib.experiment.runs.RECORDS_FILE plesty.lib.experiment.runs.DATA_DIR_VAR plesty.lib.experiment.runs.DATA_MOUNT_VAR plesty.lib.experiment.runs.PLAN_FILE plesty.lib.experiment.runs.JOURNAL_FILE plesty.lib.experiment.runs.RAW_SUBDIR plesty.lib.experiment.runs.RUN_ID_TIME_FORMAT Classes ------- .. autoapisummary:: plesty.lib.experiment.runs.Run Functions --------- .. autoapisummary:: plesty.lib.experiment.runs.run_id plesty.lib.experiment.runs.parse_run_id plesty.lib.experiment.runs.list_runs plesty.lib.experiment.runs.latest_run plesty.lib.experiment.runs.host_join plesty.lib.experiment.runs.to_local Module Contents --------------- .. py:data:: BLOB_SUBDIR :value: 'data' .. py:data:: RECORDS_FILE :value: 'records.jsonl' .. py:data:: DATA_DIR_VAR :value: 'PLESTY_DATA_DIR' .. py:data:: DATA_MOUNT_VAR :value: 'PLESTY_DATA_MOUNT' .. py:data:: PLAN_FILE :value: 'plan.json' .. py:data:: JOURNAL_FILE :value: 'journal.jsonl' .. py:data:: RAW_SUBDIR :value: 'raw' .. py:data:: RUN_ID_TIME_FORMAT :value: '%Y%m%d-%H%M%S' .. py:function:: run_id(name: str, when: Optional[datetime.datetime] = None) -> str Return the run id for experiment *name* started at *when* (now by default). .. py:function:: parse_run_id(text: str) -> Optional[tuple[str, datetime.datetime]] Split a run id into its experiment name and start time. :param text: A directory name. :returns: ``(name, started)``, or ``None`` when *text* is not a run id — the name may itself contain underscores, so the split is at the last one. .. py:function:: list_runs(run_root: str | pathlib.Path, name: Union[str, Sequence[str], None] = None) -> list[pathlib.Path] Return the run directories under *run_root*, oldest first. :param run_root: The directory runs are created in. :param name: Keep only runs of this experiment name (or any of several — an experiment that was renamed still owns its old runs). ``None`` keeps every run. .. py:function:: latest_run(run_root: str | pathlib.Path, name: Union[str, Sequence[str], None] = None) -> pathlib.Path Return the most recently started run under *run_root*. :raises FileNotFoundError: If there is no such run yet — a viewer opened before its experiment is a mistake worth naming. .. py:function:: host_join(root: str, *parts: str) -> str Join path parts in the spelling of *root*'s operating system. A data directory is spelled the way the acquiring host spells it, which is not necessarily this machine's convention — ``G:\RAWDATA`` from a Linux viewer must still become ``G:\RAWDATA\run\raw``. .. py:function:: to_local(host_path: str, data_dir: str, mount: str) -> pathlib.Path Translate a path the acquiring host reported into this machine's mount. :param host_path: The path as the acquiring host spelled it, e.g. ``G:\RAWDATA\pol_pl_20260805-091909\raw\frame.spe``. :param data_dir: The shared root as that host sees it. :param mount: The same root as this machine sees it. :raises ValueError: If *host_path* lies outside *data_dir* — a path from another rig or a stale configuration, which would otherwise resolve to a plausible-looking wrong file. .. py:class:: Run(run_dir: str | pathlib.Path, *, mount: Optional[str] = None, data_dir: Optional[str] = None, replay_rows: int = 0) Read access to one run directory: plan, journal, records, data paths. Nothing here writes. The plan is read once; the journal is replayed on each :meth:`status` because a run under way keeps appending to it. Attach to a run directory. :param run_dir: The run directory. :param mount: This machine's view of the shared data root; ``None`` takes ``PLESTY_DATA_MOUNT`` from the environment, then the mount journaled by the machine that ran the experiment (only when that directory exists here). :param data_dir: The shared root as the acquiring host sees it — only for runs that did not record it themselves. :param replay_rows: How :meth:`source` hands records out — ``0`` follows the run live, ``n`` rations a stored run *n* records per poll. .. py:attribute:: path .. py:attribute:: replay_rows :value: 0 .. py:attribute:: _mount :value: None .. py:attribute:: _data_dir :value: None .. py:attribute:: _plan :type: Optional[plesty.lib.experiment.schedule.Plan] :value: None .. py:attribute:: _started :type: Optional[dict[str, Any]] :value: None .. py:attribute:: sources :type: list[Any] :value: [] .. py:property:: id :type: str The run id (the directory name). .. py:property:: name :type: Optional[str] The experiment name the run id carries, if it parses as one. .. py:property:: started :type: Optional[datetime.datetime] The start time the run id carries, if it parses as one. .. py:property:: plan :type: plesty.lib.experiment.schedule.Plan The frozen plan (loaded once). .. py:property:: config :type: dict[str, Any] The frozen configuration; ``{}`` when the plan is unreadable. .. py:property:: journal :type: plesty.lib.experiment.journal.Journal The run's journal. .. py:method:: status() -> dict[str, Any] The run status by journal replay (see :meth:`Journal.status`). .. py:method:: records() -> Iterable[dict[str, Any]] Iterate the committed step records, oldest first. .. py:method:: _run_started() -> dict[str, Any] .. py:property:: data_dir :type: Optional[str] The shared root as the acquiring host sees it. Journaled by the run; falls back to the plan's top-level ``data_dir`` for runs written before the journal carried it, then to the ``data_dir`` given at construction. .. py:property:: raw_dir :type: Optional[str] Where the run's device-written blobs went, in the host's spelling. .. py:property:: data_mount :type: Optional[str] The shared root as this machine sees it (see the class docstring). .. py:method:: local(host_path: str) -> pathlib.Path Return *host_path* (as a record spells it) as a path on this machine. Untranslated when the run has no data root or this machine has no mount for it — the case of a viewer on the acquiring host itself. .. py:method:: source(mapper: Optional[plesty.lib.monitor.sources.FrameMapper] = None) -> Union[plesty.lib.monitor.sources.RunSource, plesty.lib.monitor.sources.ReplaySource] Return a source over this run's records — live or replayed. A :class:`~plesty.lib.monitor.sources.RunSource` when :attr:`replay_rows` is 0, else a :class:`~plesty.lib.monitor.sources.ReplaySource` handing out that many records per poll. Every source made here is kept in :attr:`sources`. Make one source per consumer: a source hands each record out once, so two views polling the same source would each see half the rows. The mapper is what they share. .. py:property:: exhausted :type: bool Whether every source made here has handed out its last record. Live sources never are (the next record may still be coming); a run without sources counts as exhausted so a watcher does not wait forever. .. py:method:: __repr__() -> str Return ``Run('')``.