Data on a Shared Disk

Some devices produce large files: a spectrometer saves a spectrum per step, a camera an image. Sending those over the network to the experiment would be slow. Instead, the device saves each file straight onto a disk the whole lab shares, and only the file’s path travels to the experiment, which records it. Large data never crosses the network twice shows this in a diagram.

A run on a single computer, whose devices save no files of their own, needs none of this page.

One disk, a different path on each computer

The shared disk — the Plesty share — is mounted under a different path on every computer, and each bench writes only into its own folder of it. With the share at /mnt/plesty on the experiment’s computer and at G:\Plesty on a Windows bench whose folder is platform-a, the same folder has two names:

	Recorded in the run	On this computer	On the bench
a device’s files	platform-a/pol-pl/<run id>/raw/spec	/mnt/plesty/platform-a/pol-pl/<run id>/raw/spec	G:\Plesty\platform-a\pol-pl\<run id>\raw\spec

So every program is told only where it sees the share, and every path that is recorded or passed between computers is relative to the share:

• the experiment reads its computer’s path from the environment variable PLESTY_DATA (see The .env File), or from data_root= when it is created;

• a device’s server is told by its bench: PLESTY_DATA, and DEVICE_DATA_ROOT, the bench’s folder, the one place it may write (see Base Device).

A relative run_root, such as pol-pl, is then taken inside the share, so the runs themselves are on it. A ~ at the start of either is this computer’s home folder: PLESTY_DATA=~/plesty is your home’s plesty folder, and a run_root of ~/runs is used as it is, not put inside the share.

Telling the devices where to save

Before setup() runs, the run’s id is known, and self.raw_dir is pol-pl/<run id>/raw. It is the same when the run is continued. In setup(), send the devices that save files there with self.route(...). Each writes into raw/<device> under its own bench’s folder, and answers with that folder relative to the share, which the run keeps:

async def setup(self) -> None:
    await super().setup()
    if self.raw_dir is not None:
        self.route(self.devices, ["spec"])

self.raw_dir is None when the experiment’s computer has no PLESTY_DATA: it could not read the files back.

Recording a device’s file

A device reports each file it saved as its own computer writes the path. The run already knows the folder, so it keeps only the file’s name: self.raw_file("spec", reported_path) returns platform-a/pol-pl/<run id>/raw/spec/<name>, which is what the step should return and have recorded.

Reading the files back, on any computer

Run.local() turns a recorded path into a path on the computer that reads it:

from plesty.lib.experiment import Run, latest_run

run = Run(latest_run("/mnt/plesty/pol-pl", name="pol_pl"))
for record in run.records():
    path = run.local(record["data_file"])   # pol-pl/<run id>/raw/spec/… → /mnt/plesty/pol-pl/…

Run.local finds the disk from, in this order: data_root= given to Run, PLESTY_DATA, and the disk path the run’s own computer recorded in the journal, if that path exists on this computer. An absolute path is returned unchanged.