plesty.lib.experiment.journal

Append-only run journal for experiment status logging and resume.

The journal is a JSON-Lines file (one event per line) that records the life of an experiment run: run_started, step_started, step_completed, step_failed, run_canceled, run_aborted, run_completed. Every append is flushed and fsynced, so the journal survives crashes; at worst the final line is torn, which Journal.replay() tolerates by skipping it.

The current run status is never stored in a mutable file — it is always derived by replaying the journal, exactly like reconstructing training progress from a deep-learning run log.

Classes

Journal

Crash-safe append-only event log for one experiment run.

Module Contents

class plesty.lib.experiment.journal.Journal(path: str | pathlib.Path)

Crash-safe append-only event log for one experiment run.

Attach the journal to path, creating parent directories.

Parameters:

path (str | pathlib.Path) – Location of the journal.jsonl file; appended to if it already exists (the resume case).

TERMINAL_EVENTS = ('run_completed', 'run_canceled', 'run_aborted')

path

append(event: str, **fields: Any) → dict[str, Any]

Append one event line and fsync it to disk.

Parameters:

• event (str) – Event name, e.g. "step_completed".

• fields (Any) – Additional payload stored on the event (step id, result path, error text, …).

Returns:

The full event record that was written.

Return type:

dict[str, Any]

replay() → list[dict[str, Any]]

Read all events back, tolerating a torn final line after a crash.

Returns:

The recorded events in write order; an empty list if the journal file does not exist yet.

Return type:

list[dict[str, Any]]

completed_steps() → set[str]

Return the ids of all steps journaled as completed.

Together with the steps that have a committed record line this is the resume set: a resumed run skips these and re-runs everything else.

Return type:

set[str]

status() → dict[str, Any]

Summarize the run status by replaying the journal.

Returns:

A dict with state ("not_started", "running", "completed", "canceled" or "aborted"), completed and failed step counts, and the last recorded event.

Return type:

dict[str, Any]