Design Decisions

Why plesty-lib is built the way it is. Each decision names the alternative that was rejected, why, and what the choice costs. To revisit a decision, add a dated note to it here rather than quietly working around it.

Independent parts instead of a chain of base classes

A device combines three parts, parameters (ConfigSystem), operations (FunctionSystem) and telemetry (TelemetrySystem), as independent base classes that BaseDeviceSyncModel inherits side by side (mixins), rather than as a chain where each builds on the previous one (ConfigurableDevice → FunctionDevice → …).

The three parts do not depend on each other, so a chain would impose an arbitrary order and make every device take every link. As separate parts, each can be tested alone (the test helpers exercise ConfigSystem by itself), and adding TelemetrySystem in 0.2.9 changed no existing device.

The cost is one class with many attributes, where names can collide. That happened: the solver attributes were renamed to _cmd_solver and _op_solver after they clashed with methods of a device (lib#1).

A description as data instead of generated text

Until 0.2.9, a device’s documentation was whatever summary() printed. doc_model() now returns a DeviceDocModel: the device’s parameters, operations and standard methods as data, with no formatting. Each consumer formats it itself.

The alternative, each consumer collecting the information from the device on its own, had already drifted: the server’s describe reply, summary() and the documentation generator each described a device slightly differently. One source also separates the methods every device has from the device’s own operations, once, for everyone.

The cost is one more step between a device and its description. summary() still builds its text the old way, so until it is moved over, two descriptions exist.

Claiming the instrument on connect, not on creation

ResourceRegistry.acquire() claims the instrument when a with block opens the device, and releases it when the block ends or the connection fails, not when the device object is created.

Claiming at creation made creating a device a side effect: documentation tools, offline tools and tests could not make a device object without claiming the instrument, and a crashed program could leave a stale claim behind. (A claim whose holder no longer exists is now reclaimed automatically.)

The cost: two objects for the same instrument now clash later, when the second one connects, and code that calls connect() directly instead of using with skips the claim altogether.

Retrying remote calls instead of failing at once

When a network client’s request times out, the reply may still arrive later and sit in the connection, and the client used to stay broken from then on (lib#10). CompositeDevice.call therefore rebuilds the client and tries again (lib#15) instead of giving up. In a measurement of several hours, one short network hiccup should not end the run, and before this every experiment wrote its own retry loop. The number of attempts is limited, and the last timeout is still raised.

The cost: a retried call may run twice, if the server received the first request but the reply was lost. For an operation that must not run twice, pass retries=1.

Telemetry on request instead of a background service

TelemetrySystem defines what telemetry looks like (DeviceStatus, TelemetryEvent) and lets a consumer register callbacks. It runs no thread, polls nothing and sends nothing anywhere (lib#4).

A built-in telemetry service was rejected because it would force its own threads and transport on every device, and polling in the background can upset instruments that are sensitive to timing. With no callback registered, telemetry costs nothing: not even the event object is created.

The cost: whoever wants regular updates runs their own loop, and a slow callback slows down the device call that triggers it.

A journal file instead of a database

An experiment’s state is a journal to which lines are only ever added, next to a fixed plan.json (lib#3).

A database would allow queries and several writers, but a measurement has one writer, and a database server (or even the file locking of a database file on a network drive) is exactly what fails in the middle of the night in an instrument room. An append-only file survives a crash (an incomplete last line is detected and skipped), and can be read, compared and searched with ordinary tools. The plan’s fingerprint turns “the plan was changed” into an error on resume instead of a corrupted run.

The cost: reading a run’s state means reading its whole journal, which is fine for thousands of steps and would not be for millions.

Original files plus a description, with HDF5 as an option

save_result keeps each result in its original format (a .npy array, the camera’s own image file, or small values inside the JSON description), described by a ResultDocument. convert_to_hdf5 packs results into one HDF5 file when someone wants that.

Storing everything in HDF5 from the start was rejected: it would make h5py a requirement of every device module, convert images that are already valid PNG files, and put a whole run at risk in one file. Writing the data file first and its description second makes each result safe on its own.

The cost: a run is many small files rather than one archive, and whoever wants HDF5 runs the conversion.