plesty.sdk.report ================= .. py:module:: plesty.sdk.report .. autoapi-nested-parse:: The report artifacts a module publishes about its own quality. A module publishes two kinds of document under ``reports/``, sharing one envelope so a consumer reads either the same way:: reports/ check.json plesty check committed / field-test.json field test committed *.md every report git-ignored *.jsonl raw per-run records git-ignored ``check.json`` sits at the top because the standard gates are a property of the module and have no instrument. ``field-test.json`` is written by ``plesty.lib.test.field_test`` after a run against real hardware, and belongs to a module *and* the hardware it ran against — hence the directory. This module owns the ``check`` half and *reads* the field-test half. The two are written by different tools at different times, which is the reason they are two files: a merged one would be a write race between ``plesty check`` and a hardware run. Only the markdown and JSONL forms carry the host, the address and the driver's own error messages, and the scaffold git-ignores them. Everything here is either derived from the module's own source — already public — or is a gate result. Attributes ---------- .. autoapisummary:: plesty.sdk.report.SCHEMA_VERSION plesty.sdk.report._TOOLING_PACKAGES plesty.sdk.report.REPORTS_DIRNAME plesty.sdk.report.GATE_CATALOG Exceptions ---------- .. autoapisummary:: plesty.sdk.report.ReportError Classes ------- .. autoapisummary:: plesty.sdk.report.GateRecord plesty.sdk.report.CheckReport Functions --------- .. autoapisummary:: plesty.sdk.report.tooling_info plesty.sdk.report.module_info plesty.sdk.report.read_field_test_reports plesty.sdk.report.field_latencies plesty.sdk.report._keep_slowest plesty.sdk.report.error_findings plesty.sdk.report._require_known_version plesty.sdk.report._module_version plesty.sdk.report._repository_path plesty.sdk.report._git plesty.sdk.report._utc_now Module Contents --------------- .. py:data:: SCHEMA_VERSION :value: '1.1' .. py:data:: _TOOLING_PACKAGES :value: ('plesty-sdk', 'plesty-lib') .. py:data:: REPORTS_DIRNAME :value: 'reports' .. py:data:: GATE_CATALOG :type: dict[str, tuple[str, str, str]] .. py:exception:: ReportError Bases: :py:obj:`RuntimeError` A published report is missing, malformed, or from a contract we cannot read. Initialize self. See help(type(self)) for accurate signature. .. py:class:: GateRecord One gate's outcome, as a published page shows it. :ivar key: Gate key used by ``plesty check``. :ivar number: Gate number or letter code, e.g. ``"7"`` or ``"d1"``. :ivar name: Human-readable gate title. :ivar tier: Which family the gate belongs to — ``sdk`` or a module type. :ivar status: One of :data:`PASS`, :data:`FAIL`, :data:`SKIP`. :ivar reason: Why it skipped or failed. ``None`` when it passed. .. py:attribute:: key :type: str .. py:attribute:: number :type: str .. py:attribute:: name :type: str .. py:attribute:: tier :type: str .. py:attribute:: status :type: str .. py:attribute:: reason :type: str | None :value: None .. py:method:: to_dict() -> dict[str, Any] Return a JSON-serialisable mapping of the gate record. .. py:class:: CheckReport The standard gates, as ``check.json``. Every module has one: the standard gates run without an instrument. That is the difference from a field test report, whose absence is meaningful and is rendered as absence rather than as an empty section. :ivar module: Module identification — name, version, type, standard, repository. :ivar standard: The compliance standard the run enforced. :ivar gates: Gate outcomes in execution order. :ivar coverage_percent: Measured test coverage, when gate 7 ran. :ivar coverage_required: The threshold gate 7 enforces. .. py:attribute:: module :type: dict[str, Any] .. py:attribute:: standard :type: str .. py:attribute:: gates :type: list[GateRecord] :value: [] .. py:attribute:: coverage_percent :type: float | None :value: None .. py:attribute:: coverage_required :type: float | None :value: None .. py:method:: record(key: str, status: str, reason: str | None = None) -> None Append one gate outcome. :param key: Gate key; unknown keys are recorded under their own name rather than dropped, so a gate added without a catalog entry is visible instead of silently missing. :param status: One of :data:`PASS`, :data:`FAIL`, :data:`SKIP`. :param reason: Why it skipped or failed. .. py:method:: record_missing_as_skipped(keys: tuple[str, Ellipsis], reason: str) -> None Record every gate in *keys* that has no outcome yet. A run that stopped at the first failure must not read as a clean sweep of a shorter sequence — the gates that never ran say so, with why. :param keys: The full gate sequence for this standard. :param reason: Why the remaining gates did not run. .. py:property:: counts :type: dict[str, int] Number of gates per status. .. py:property:: ok :type: bool ``True`` when no gate failed. A skipped gate does not fail a run. .. py:method:: to_dict() -> dict[str, Any] Return the document, sharing its envelope with ``field-test.json``. .. py:method:: write(project_dir: pathlib.Path) -> pathlib.Path Write ``/reports/check.json``. :param project_dir: Root directory of the module. :returns: The path written. .. py:function:: tooling_info() -> dict[str, str] Return the versions of the packages that decide what a run checked. A green ``check.json`` says which gates passed but not what they were — the gate set, and what each gate does, is a property of the SDK and of plesty-lib, and both change. Two reports claiming "Device API Pipeline" can mean two different contracts (v0.4.0 dropped the two hardware gates, made the parameter and operation gates able to fail at all, and moved the schema gate onto ``doc_model()``). Without this a reader cannot tell which of those a report is asserting. :returns: Package name to installed version, for each package that is installed. An absent package is left out rather than recorded as unknown. .. py:function:: module_info(project_dir: pathlib.Path, pyproject_toml: dict[str, Any], standard: str) -> dict[str, Any] Build the shared ``module`` block of a report envelope. :param project_dir: Root directory of the module. :param pyproject_toml: Parsed ``pyproject.toml`` content. :param standard: The compliance standard resolved for this run. :returns: Name, version, module type, standard and repository path. .. py:function:: read_field_test_reports(project_dir: pathlib.Path) -> list[tuple[pathlib.Path, dict[str, Any]]] Read every published ``field-test.json`` under the module's ``reports/``. A module that has never been field-tested publishes none, and that is the honest rendering: "never run" and "ran and found nothing" are different facts, and a placeholder invites a reader to treat them as the same. :param project_dir: Root directory of the module. :returns: ``(path, document)`` pairs, sorted by path. :raises ReportError: If a document is unreadable, is not an object, or declares a schema version this SDK does not know. .. py:function:: field_latencies(project_dir: pathlib.Path) -> dict[str, float] Return the measured call time of everything a field test reached. Keyed by parameter key and operation name, in milliseconds, so the device documentation can label what was exercised on hardware with what it cost. A parameter's figure is its query; an operation's is the call. Untested entries are absent rather than zero. A label on every entry is a label nobody reads, and one that appears is then a thing that happened rather than one that changed colour. Where several instruments reached the same entry the slowest is kept. Two reports are two rigs, and documentation that promised the faster one would be read as a guarantee by whoever has the other. :param project_dir: Root directory of the module. :returns: Name → median milliseconds. Empty when nothing was ever field-tested. .. py:function:: _keep_slowest(latencies: dict[str, float], name: Any, calls: Any) -> None Record *calls*' median against *name*, keeping the slowest seen. :param latencies: The mapping being built, updated in place. :param name: Parameter key or operation name. :param calls: The call record, or ``None`` when the entry was never reached. .. py:function:: error_findings(document: dict[str, Any]) -> list[dict[str, Any]] Return the ``error`` findings of a field test document. An error finding means the module declares an API the instrument does not have — it is telling the rig, a remote client and the manager's API summary that something exists which does not. That is worth failing a publish over, which is what gate f1 does with this list. :param document: A parsed ``field-test.json``. .. py:function:: _require_known_version(path: pathlib.Path, document: dict[str, Any]) -> None Refuse a document whose major version this SDK cannot render. :param path: Where the document came from, for the error message. :param document: The parsed document. :raises ReportError: If the version is absent or of an unknown major. .. py:function:: _module_version(name: str, project: dict[str, Any], project_dir: pathlib.Path) -> str Resolve the module's version from its metadata or its git history. :param name: Distribution name. :param project: The ``[project]`` table. :param project_dir: Root directory of the module, for the git fallback. .. py:function:: _repository_path(project_dir: pathlib.Path) -> str | None Return the origin remote's path without its host, or ``None``. The path is what every PLESTY surface links to and is public by construction; the host is dropped because it identifies an instance, not a module. :param project_dir: Root directory of the module. .. py:function:: _git(project_dir: pathlib.Path, *args: str) -> str Run a read-only git command in *project_dir*, returning its output or ``""``. .. py:function:: _utc_now() -> str Return the current UTC time as an ISO-8601 string with second precision.