`statcatlib.bundle`¶

Generated from the docstrings.

bundle ¶

The statcat section of an sdmxlib edition manifest, and its check.

sdmxlib owns manifest.json (sdmxlib.edition): the edition id, a digest per file, and the checker. A statcat edition is an sdmxlib edition plus the statcat schema and search/, so statcatlib adds what only it can describe: a statcat section in the manifest, written by catalog_section when WorkingCopy.seal seals, and check_catalog, which checks that the catalog and its search index are what that section says (GH-3). Run check_catalog as an extension of sdmxlib.edition.check_edition.

It also holds the catalog description's JSON form: description_to_json and description_from_json convert a CatalogDescription's catalog-level fields for the Inventory root's extensions.statcat, which CatalogAdmin.describe writes and Catalog.description reads.

The module keeps its name from the 0.11 bundle format; it describes editions.

catalog_section ¶

catalog_section(
    store: _Store, *, kind: Kind = "data"
) -> dict[str, object]

Describe a working copy's edition, as its manifest's statcat section (format 3).

The section records the product's kind, the number of catalog datasets and, when a search index is built, its index format version, embedding model, each language's row count and stemmer, and the annotations it was built from. WorkingCopy.seal computes it after rebuilding the index and before closing the store, and passes it to sdmxlib.edition.seal as extensions={"statcat": section}.

Parameters:

Name	Type	Description	Default
`store`	`_Store`	The working copy's store, open.	required
`kind`	`Kind`	The product's kind, `data` or `structure`.	`'data'`

Returns:

Type	Description
`dict[str, object]`	The section, ready for `seal(extensions=...)`.

structure_content ¶

structure_content(
    conn: DuckDBPyConnection,
) -> list[Finding]

What a structure product's edition may not hold: each dataflow, binding and partial in its store.

A structure product holds full artefacts only. check_catalog runs this on a sealed structure edition, and WorkingCopy.seal on a structure working copy before anything is sealed.

Parameters:

Name	Type	Description	Default
`conn`	`DuckDBPyConnection`	A connection whose current catalog is the edition's, so `sdmx.artefact` names its table.	required

Returns:

Type	Description
`list[Finding]`	One finding per such artefact, by URN.

check_catalog ¶

check_catalog(
    root: Path,
    manifest: Manifest,
    reg: EditionRegistry,
    profile: Profile,
) -> list[Finding]

The statcat check, an extension of sdmxlib.edition.check_edition.

Pass it as check_edition(path, at=..., extensions={STATCAT_SECTION: check_catalog}). It checks that:

the statcat schema is in the edition's store at the version this build reads, and the manifest has a statcat section in a format this build reads;
the section's kind is one of PRODUCT_KINDS, and a structure edition binds no dataflow to observations and holds no dataflow, binding or partial (structure_content);
the catalog holds the datasets the section counts, and every dataset linked to a dataflow names one the registry holds;
search/ is present exactly when the section declares an index; index.json records a version this build reads, the one the section declares; the index has the declared languages, each stemmed as declared and holding the declared rows, and no table that index.json does not name;
the store's annotations of the types the index read are the ones it was built from, so an overlay folded in since shows up as a stale index.

Parameters:

Name	Type	Description	Default
`root`	`Path`	The edition directory.	required
`manifest`	`Manifest`	Its manifest.	required
`reg`	`EditionRegistry`	sdmxlib's registry over the edition.	required
`profile`	`Profile`	The `check_edition` profile that runs it. Every profile runs the same checks.	required

Returns:

Type	Description
`list[Finding]`	Everything found wrong with the catalog's part of the edition.

overlay_affects_search ¶

overlay_affects_search(
    overlay: Overlay, types: Collection[str] | None = None
) -> tuple[OverlayEntry, ...]

The entries of overlay that change what the search index should hold.

An overlay applied at serve time never reaches statcatlib's index, and one folded into an edition needs build_index() before search reflects it. Either way, an entry acting on an annotation type the index reads is a change search will not see on its own. Call this where overlays are loaded, to log or refuse them. Static: it needs no edition.

Parameters:

Name	Type	Description	Default
`overlay`	`Overlay`	The overlay document.	required
`types`	`Collection[str] \| None`	The annotation types the index reads. `None` means `SEARCH_ANNOTATION_TYPES`, what `Catalog.build_index` reads by default; pass `cat.index_annotations["types"]` for an index built otherwise.	`None`

Returns:

Type	Description
`tuple[OverlayEntry, ...]`	Those entries, in document order; empty, and so falsy, when there are none.

description_to_json ¶

description_to_json(
    description: CatalogDescription,
) -> dict[str, object]

The catalog-level fields of description, as the JSON the Inventory root's extensions.statcat holds.

The fields are identifier, title, description, publisher, homepage, language, licence, theme_taxonomy, issued and modified. Multilingual text becomes an object of language tag to text, and an instant an ISO 8601 string. datasets, services, records, spatial_coverage and temporal_coverage are left out: the datasets and services come from the live editions, and nothing stores them twice.

Parameters:

Name	Type	Description	Default
`description`	`CatalogDescription`	The description.	required

Returns:

Type	Description
`dict[str, object]`	A JSON object, which `description_from_json` reads back.

description_from_json ¶

description_from_json(
    data: Mapping[str, object],
) -> CatalogDescription

The CatalogDescription that description_to_json wrote.

Keys it does not know are ignored, so a section a later statcatlib extends still reads. The fields description_to_json leaves out take their defaults.

Parameters:

Name	Type	Description	Default
`data`	`Mapping[str, object]`	The `description` object of the root's `extensions.statcat`.	required

Returns:

Type	Description
`CatalogDescription`	The description.

Raises:

Type	Description
`TypeError`	A field has the wrong JSON type, or `identifier`, `title`, `description` or `publisher` is missing.

description_from_root ¶

description_from_root(
    extensions: Mapping[str, Mapping[str, object]],
) -> CatalogDescription | None

The catalog description an Inventory root's extensions hold, or None when it holds none.

Parameters:

Name	Type	Description	Default
`extensions`	`Mapping[str, Mapping[str, object]]`	The root's `extensions`.	required

Returns:

Type	Description
`CatalogDescription \| None`	The description in `extensions.statcat.description`, read with
`CatalogDescription \| None`	`description_from_json`.

statcatlib.bundle¶

bundle ¶

catalog_section ¶

structure_content ¶

check_catalog ¶

overlay_affects_search ¶

description_to_json ¶

description_from_json ¶

description_from_root ¶

`statcatlib.bundle`¶