Data & Metadata Schemas

plesty-lib defines one schema contract per data domain so that every HUB module acquires, stores, and exchanges data in the same shape. Three domains are covered (issue lib#4):

Domain	Schema	Module
Device configuration	ConfigParameter / ConfigGroup	plesty.lib.device.params
Operational telemetry	DeviceStatus / TelemetryEvent	plesty.lib.device.telemetry
Experimental output	ResultDocument	plesty.lib.data.io

Device configuration

Structured representation of device parameters and settings — see Parameter System. Every parameter is a ConfigParameter (name, dtype, unit, constraints, command binding) organised into ConfigGroups and managed by the ConfigSystem that every device model composes. Device modules declare their parameters in a YAML/JSON schema which ConfigSystem materialises at init.

Operational telemetry

Real-time status, state, and diagnostic data, defined in plesty.lib.device.telemetry. Two shapes cover pull and push:

• DeviceStatus — a point-in-time snapshot. Every device inherits status() via the TelemetrySystem mixin; the default implementation is assembled from the connection check and the cached configuration values (no device I/O), so it is always cheap to call:

  status = device.status()
  status.connection   # "connected" | "disconnected" | "connecting" | "error"
  status.state        # "ready", "busy", ... (override device_state() to refine)
  status.parameters   # last-known parameter values
  status.last_error   # most recent error message, if any
  

• TelemetryEvent — a timestamped record of kind reading, state_change, error, or warning. Consumers register a hook; the base device emits state_change/reading events automatically on every successful write/query, and devices can emit custom events with emit_telemetry():

  events = []
  device.register_telemetry_hook(events.append)
  device.write("wavelength", 850.0)   # emits a state_change event
  
  from plesty.lib.device.telemetry import append_events
  append_events(run_dir / "telemetry.jsonl", events)
  

Hooks run synchronously and never disturb device operation (a failing hook is logged and skipped). When no hook is registered, no event is constructed. append_events/read_events persist events in the same crash-tolerant append-only JSONL format as the experiment journal. Polling loops, threads, and transports are deliberately out of scope — they belong to monitors and services, not to the schema.

Experimental output

Acquired data plus its metadata, defined in plesty.lib.data.io. The storage contract keeps blobs in their original raw format (.npy for PlestyArray, native encoding for image bytes) described by a JSON metadata document — the typed ResultDocument model — holding the Plesty meta (name, unit, range, description), timestamps, provenance, and a reference link to the blob:

from plesty.lib.data import save_result, load_result, load_document

path = save_result(array, run_dir / "step_0001", provenance={"step_id": "s1"})
document = load_document(path)   # metadata only — no blob I/O
result = load_result(path)       # reconstructed PlestyArray

The JSON document is the commit record: a result exists once its .json file does, and writes are crash-safe (blob first, document last, both atomic). convert_to_hdf5 optionally packs a set of documents into a single HDF5 archive for exchange (requires the optional h5py dependency).