plesty.server.model.experiment

One experiment as the fleet declares it, and what its module says it takes.

A device and an experiment are declared almost alike — a package, a version or a repository and ref, arguments and an environment — and installed by exactly the same path, which is keyed by the package and knows nothing of devices. The differences are what this module is for:

• An experiment binds no port and is not a daemon. There is no port and no autostart: nothing probes it for liveness and fleet up does not start it. It runs when an operator runs it, once, to the end of its plan.

• It is installed from its repository, not from a release. Its config/*.yaml and .env.example are in the checkout and not in the wheel, so a release install would leave nothing to configure.

• It declares what it takes. python -m <module> --manifest prints the document Manifest describes, and the console builds its settings from that rather than from anything written down here. The manifest is read from the module, never cached into the fleet: a fleet that carried a copy would be a second declaration to keep in step.

Attributes

_KNOWN_KEYS
DEFAULT_CONFIG

Exceptions

ManifestError

Raised when a module's manifest cannot be read or makes no sense.

Classes

ExperimentSpec	One experiment as the fleet declares it.
Parameter	One configuration parameter the module declares.
ExperimentManifest	What a module answered when asked what it takes.

Functions

_parameters(→ list[Parameter])	Flatten a manifest's grouped schema into parameters.
_environment(→ list[tuple[str, str, bool]])	Read the manifest's environment declarations.
_number(→ float | None)	Return value as a float, or None when it is not one.

Module Contents

plesty.server.model.experiment._KNOWN_KEYS

plesty.server.model.experiment.DEFAULT_CONFIG = 'config/default.yaml'

class plesty.server.model.experiment.ExperimentSpec

One experiment as the fleet declares it.

Variables:

• name – Instance name — the key its runs and jobs are recorded under; unique per bench.

• package – PyPI distribution of the experiment module.

• version – Pinned release, or None for the newest at install time.

• git – Repository URL to install from; an experiment is normally installed this way, since its configuration is in the checkout.

• ref – Branch, tag or commit for a git source; None = v<version> when a version is given, else the default branch.

• path – A local repository to run instead of an install — the folder Hub ▸ Load from… loaded. Only a declaration naming it runs it.

• module – Import path run with python -m; derived from package.

• config – Configuration file, relative to the checkout.

• args – Arguments appended to python -m <module>, beyond the ones the console supplies itself (--config, --run-root, --resume).

• env – Environment set for the run on top of the bench’s own.

name: str

package: str

version: str | None = None

git: str | None = None

ref: str | None = None

path: str | None = None

module: str = ''

config: str = 'config/default.yaml'

args: tuple[str, Ellipsis] = ()

env: collections.abc.Mapping[str, str]

__post_init__() → None

Fill the derived fields and validate the declaration.

Raises:

FleetError – If the name is unusable as a directory key, or no package is named.

Return type:

None

property git_ref: str | None

The ref a git install checks out, or None for the default branch.

Return type:

str | None

property requirement: str

The pip requirement a release install resolves.

Return type:

str

command(python: str, run_root: str, resume: str | None = None, config: str | None = None) → list[str]

Build the command that runs this experiment.

The console supplies the three arguments it is entitled to decide — which configuration, where the run goes, and whether this continues an earlier one — and the declaration’s own arguments follow, so a module option always wins over nothing and never over the operator.

Parameters:

• python (str) – Interpreter of the module’s environment.

• run_root (str) – Directory the run’s checkpoint directory goes in.

• resume (str | None) – Run id to continue, or None for a new run.

• config (str | None) – The configuration file, when the run has its own; the declared one by default. Passed once: a second --config would win only by coming last.

Returns:

The command line.

Return type:

list[str]

manifest_command(python: str) → list[str]

Build the command that asks the module what it takes.

Parameters:

python (str) – Interpreter of the module’s environment.

Returns:

The command line; it connects to nothing and needs no configuration.

Return type:

list[str]

devices_command(python: str, config: str) → list[str]

Build the command that asks which devices a run of config uses.

Parameters:

• python (str) – Interpreter of the module’s environment.

• config (str) – The configuration file the run will read.

Returns:

The command line; it connects to nothing.

Return type:

list[str]

classmethod from_mapping(name: str, raw: collections.abc.Mapping[str, Any]) → ExperimentSpec

Build a spec from one experiments: entry.

Parameters:

• name (str) – The instance name the entry is keyed by.

• raw (collections.abc.Mapping[str, Any]) – The entry.

Raises:

FleetError – On unknown keys or a malformed value.

Return type:

ExperimentSpec

to_mapping() → dict[str, Any]

The declaration’s YAML form, derived defaults left out.

Return type:

dict[str, Any]

exception plesty.server.model.experiment.ManifestError

Bases: RuntimeError

Raised when a module’s manifest cannot be read or makes no sense.

Initialize self. See help(type(self)) for accurate signature.

class plesty.server.model.experiment.Parameter

One configuration parameter the module declares.

The fields are the device vocabulary — the console renders an experiment’s settings with the same code that renders a device’s panel, so nothing here is new and nothing here may drift from it.

Variables:

• key – How the configuration addresses it, group.parameter for a grouped one.

• group – The group it belongs to.

• name – Its name within the group.

• dtype – Declared type name; yaml for a value that is a document rather than a scalar.

• unit – Physical unit, or "" for a count, or None.

• description – What it means.

• default – What the module uses when the configuration omits it.

• minimum – Lower bound, or None.

• maximum – Upper bound, or None.

• options – Allowed values, which make it a choice.

key: str

group: str

name: str

dtype: str = 'str'

unit: str | None = None

description: str = ''

default: Any = None

minimum: float | None = None

maximum: float | None = None

options: tuple[Any, Ellipsis] | None = None

property is_document: bool

Whether its value is a document rather than a scalar.

Return type:

bool

class plesty.server.model.experiment.ExperimentManifest

What a module answered when asked what it takes.

Variables:

• module – Import path the manifest names.

• title – Human title of the experiment.

• config – The configuration file the module uses by default.

• run_root – Where the module puts runs when nothing overrides it.

• viz – Import path of its viewer, or None when it ships none.

• parameters – Every declared configuration parameter, in file order.

• environment – The variables the run reads, each with whether it is required and what it is for.

• chooses_devices – Whether the module names the devices of its setup.yaml a configuration uses (--devices); without it every device is served (plesty-lib#62).

module: str

title: str = ''

config: str = 'config/default.yaml'

run_root: str = 'runs'

viz: str | None = None

parameters: tuple[Parameter, Ellipsis] = ()

environment: tuple[tuple[str, str, bool], Ellipsis] = ()

chooses_devices: bool = False

property groups: list[str]

The parameter groups, in the order they were declared.

Return type:

list[str]

in_group(group: str) → list[Parameter]

The parameters of group, in declaration order.

Parameters:

group (str)

Return type:

list[Parameter]

property required_env: list[str]

The environment variables a run cannot do without.

Return type:

list[str]

missing_env(values: collections.abc.Mapping[str, str]) → list[str]

The required variables values does not set, or sets empty.

Parameters:

values (collections.abc.Mapping[str, str]) – The environment the run would get.

Returns:

The missing variable names, in declaration order. A run can be refused on these before anything is started, rather than aborting part-way through a plan.

Return type:

list[str]

classmethod parse(text: str) → ExperimentManifest

Read a manifest from what --manifest printed.

Parameters:

text (str) – The command’s standard output.

Returns:

The manifest.

Raises:

ManifestError – If the output is not the document it should be.

Return type:

ExperimentManifest

plesty.server.model.experiment._parameters(schema: collections.abc.Mapping[str, Any]) → list[Parameter]

Flatten a manifest’s grouped schema into parameters.

A malformed entry is skipped rather than raising: the console is showing what a module said about itself, and one unreadable parameter is not a reason to offer none of the others.

Parameters:

schema (collections.abc.Mapping[str, Any]) – The manifest’s schema block.

Returns:

The parameters, groups in declaration order.

Return type:

list[Parameter]

plesty.server.model.experiment._environment(raw: Any) → list[tuple[str, str, bool]]

Read the manifest’s environment declarations.

Parameters:

raw (Any) – The manifest’s environment block.

Returns:

One (name, description, required) per readable entry.

Return type:

list[tuple[str, str, bool]]

plesty.server.model.experiment._number(value: Any) → float | None

Return value as a float, or None when it is not one.

Parameters:

value (Any)

Return type:

float | None