solprop.recipe.document#

The recipe document: its tables, its format version and the forms of its values.

A recipe is a TOML document with five fixed tables, and no other top-level name, so a future keyword can never collide with the format:

[recipe]: format (an integer) and route (a string), required; title and solprop (the version that wrote it), informative strings.
[build]: The keywords of the route’s Simulation.from_* constructor.
[execution]: fields, field_planes and resume_history.
[pins]: Values set from a design target.
[scan]: A scan over parameter paths.

This layer knows the document’s shape and nothing of a route’s keywords: those, and which form each value takes, are the parameter metadata’s. It reads text with tomllib, checks the tables and the format version, and writes canonical text with tomli_w. Every value is a string, a boolean, a 64-bit integer, a float, a table or an array: TOML’s dates and times are refused, and so is an integer outside 64 bits, which a recipe writes as a decimal string (encode_integer()), since other TOML readers refuse it.

The decoders and encoders here are the value forms the metadata calls: the "none" spelling, complex numbers, large integers, numeric arrays and data files.

Module Attributes

`FORMAT`	The recipe format this solprop writes, and the newest it reads.
`TABLES`	The top-level tables, in the order they are written.
`RECIPE_KEYS`	The keys of `[recipe]`, in the order they are written; the first two required.
`EXECUTION_KEYS`	The keys of `[execution]`: the run's settings that are not build keywords.
`REFUSED_BUILD_KEYS`	Keywords a recipe may not hold: the caller or the command line supplies them.
`MAX_ARRAY_LENGTH`	The longest array a recipe holds inline; a longer one is a data file.
`NONE`	The spelling of `None` for a keyword whose default is not `None`.
`Document`	A recipe document: each present top-level table, as `tomllib` reads it.

Functions

`array_from_bytes`(data, reference, *[, path])	Load a numeric array from the bytes of a `.npy` file, with pickling disabled.
`checked_bytes`(data, reference, *[, path])	Return a data file's bytes held in memory, checking its SHA-256 digest.
`decode_array`(value, decode_element, *[, ...])	Read a one-dimensional array held inline, element by element.
`decode_complex`(value, *[, path])	Read a complex number written as `{re = ..., im = ...}`.
`decode_data_file`(value, *[, path])	Read an array's data-file reference, `{file = ..., sha256 = ...}`.
`decode_integer`(value, *[, path])	Read an integer, given as a TOML integer or a decimal string.
`decode_none`(value)	Read the `"none"` spelling of `None`.
`dump_text`(document)	Write a recipe document as canonical TOML text.
`encode_complex`(value)	Write a complex number as `{re, im}`, the inverse of `decode_complex()`.
`encode_data_file`(reference)	Write a data-file reference, the inverse of `decode_data_file()`.
`encode_integer`(value)	Write an integer as TOML holds it, the inverse of `decode_integer()`.
`encode_none`(value)	Write `None` as `"none"`, the inverse of `decode_none()`.
`load_array`(reference, base_dir, *[, ...])	Load a numeric array from a `.npy` data file, with pickling disabled.
`parse_text`(text)	Read a recipe's TOML text into a checked document of this format.
`read_data_file`(reference, base_dir, *[, ...])	Read a data file's bytes, checking its SHA-256 digest.
`resolve_data_file`(reference, base_dir, *[, ...])	Return the real path of a data file, refusing one outside the recipe's directory.
`sha256_digest`(data)	Return the SHA-256 digest a data-file reference records for `data`.
`unknown_key`(key, valid, *, where[, path])	Return the error for an unknown key, naming the nearest valid one.
`upgrade`(document)	Return a document upgraded from its format to `FORMAT`.
`validate_document`(document)	Check a recipe document's tables and format, and upgrade it to this format.

Classes

DataFile(file, sha256)

A reference to a data file beside the recipe, with its SHA-256 digest.

class solprop.recipe.document.DataFile(file, sha256)[source]#

Bases: object

A reference to a data file beside the recipe, with its SHA-256 digest.

Attributes:

filestr: The path, relative to the recipe’s directory.
sha256str: The SHA-256 digest of the file’s bytes, as 64 hexadecimal digits; stored in lower case.

Raises:

RecipeError: If file is empty or absolute, or sha256 is not 64 hexadecimal digits.

Examples

>>> DataFile("table.npy", 64 * "A").sha256 == 64 * "a"
True

file: str#

sha256: str#

type solprop.recipe.document.Document = dict[str, dict[str, object]]#: A recipe document: each present top-level table, as tomllib reads it.

solprop.recipe.document.EXECUTION_KEYS = ('fields', 'field_planes', 'resume_history')#: The keys of [execution]: the run’s settings that are not build keywords.

solprop.recipe.document.FORMAT = 1#: The recipe format this solprop writes, and the newest it reads.

solprop.recipe.document.MAX_ARRAY_LENGTH = 10000#: The longest array a recipe holds inline; a longer one is a data file.

solprop.recipe.document.NONE = 'none'#: The spelling of None for a keyword whose default is not None.

solprop.recipe.document.RECIPE_KEYS = ('format', 'route', 'title', 'solprop')#: The keys of [recipe], in the order they are written; the first two required.

solprop.recipe.document.REFUSED_BUILD_KEYS = ('output', 'progress', 'progress_period', 'resume', 'stop', 'snapshots', 'tdse_archive')#: Keywords a recipe may not hold: the caller or the command line supplies them.

solprop.recipe.document.TABLES = ('recipe', 'build', 'execution', 'pins', 'scan')#: The top-level tables, in the order they are written.

solprop.recipe.document.array_from_bytes(data, reference, *, path=None)[source]#

Load a numeric array from the bytes of a .npy file, with pickling disabled.

Parameters:

databytes: The file’s checked contents.
referenceDataFile: The reference, for the errors.
pathstr or None, optional: The parameter path, for the errors.

Returns:

numpy.ndarray: The array, of a boolean, integer, float or complex dtype.

Raises:

RecipeError: If it is not a .npy file, it holds objects (which only pickling could load) or its dtype is not numeric.

solprop.recipe.document.checked_bytes(data, reference, *, path=None)[source]#

Return a data file’s bytes held in memory, checking its SHA-256 digest.

The bytes of a file a run’s output embedded (SEP 41, S2) are checked as a file’s on disk are.

Parameters:

databytes: The contents.
referenceDataFile: The reference that records the digest.
pathstr or None, optional: The parameter path, for the error.

Returns:

bytes: data.

Raises:

RecipeError: If the digest is not the one recorded.

solprop.recipe.document.decode_array(value, decode_element, *, path=None, max_length=10000)[source]#

Read a one-dimensional array held inline, element by element.

Parameters:

valueobject: A list of at most max_length elements, none of them a list.
decode_elementcallable: Called as decode_element(element, index, element_path) for each element, where element_path is path[index] (or None), so that the element’s unit can depend on its index; it returns the decoded element.
pathstr or None, optional: The parameter path of the array.
max_lengthint, optional: The longest array held inline.

Returns:

list: The decoded elements.

Raises:

RecipeError: If value is not a list, is longer than max_length (a data file holds it instead) or has a list as an element; and whatever decode_element raises.

Examples

>>> decode_array([1, 2], lambda element, index, path: 10 * element, path="x")
[10, 20]

solprop.recipe.document.decode_complex(value, *, path=None)[source]#

Read a complex number written as {re = ..., im = ...}.

Parameters:

valueobject: A table with exactly the keys re and im, each a number.
pathstr or None, optional: The parameter path, for the error.

Returns:

complex: The number.

Raises:

RecipeError: If value is not such a table.

Examples

>>> decode_complex({"re": 1.45, "im": 0.0})
(1.45+0j)

solprop.recipe.document.decode_data_file(value, *, path=None)[source]#

Read an array’s data-file reference, {file = ..., sha256 = ...}.

Parameters:

valueobject: A table with exactly the string keys file and sha256.
pathstr or None, optional: The parameter path, for the error.

Returns:

DataFile: The reference; the file is not read.

Raises:

RecipeError: If value is not such a table.

Examples

>>> decode_data_file({"file": "betas.npy", "sha256": 64 * "0"}).file
'betas.npy'

solprop.recipe.document.decode_integer(value, *, path=None)[source]#

Read an integer, given as a TOML integer or a decimal string.

The string form holds an integer that TOML’s 64 bits cannot, such as a drawn 128-bit noise seed; any integer keyword accepts it.

Parameters:

valueobject: An int (not a boolean, and not a float even if integral) or a string of decimal digits with an optional sign.
pathstr or None, optional: The parameter path, for the error.

Returns:

int: The integer.

Raises:

RecipeError: If value is neither.

Examples

>>> decode_integer("340282366920938463463374607431768211455")
340282366920938463463374607431768211455

solprop.recipe.document.decode_none(value)[source]#

Read the "none" spelling of None.

Only a keyword whose default is not None takes the spelling; the metadata says which.

Parameters:

valueobject: The recipe value.

Returns:

object or None: None for the string "none", and otherwise value itself.

Examples

>>> decode_none("none") is None
True
>>> decode_none(0.5)
0.5

solprop.recipe.document.dump_text(document)[source]#

Write a recipe document as canonical TOML text.

The tables are written in the order of TABLES and [recipe]’s keys in the order of RECIPE_KEYS; every other table keeps its own order. Keys that are not bare TOML keys, such as parameter paths in [pins], are quoted. No comments are written.

Parameters:

documentMapping: The document, as parse_text() returns it; every value already encoded (a complex number as a table, a wide integer as a decimal string).

Returns:

str: The text, which parse_text() reads back to an equal document.

Raises:

RecipeError: If the document is refused by validate_document().

Examples

>>> recipe = {"route": "gnlse", "format": 1}
>>> print(dump_text({"build": {"fwhm": "30 fs"}, "recipe": recipe}))
[recipe]
format = 1
route = "gnlse"

[build]
fwhm = "30 fs"

solprop.recipe.document.encode_complex(value)[source]#

Write a complex number as {re, im}, the inverse of decode_complex().

Parameters:

valuecomplex: The number.

Returns:

dict: {"re": value.real, "im": value.imag}.

Examples

>>> encode_complex(1.45 + 2e-6j)
{'re': 1.45, 'im': 2e-06}

solprop.recipe.document.encode_data_file(reference)[source]#

Write a data-file reference, the inverse of decode_data_file().

Parameters:

referenceDataFile: The reference.

Returns:

dict: {"file": ..., "sha256": ...}.

Examples

>>> encode_data_file(DataFile("betas.npy", 64 * "0"))["file"]
'betas.npy'

solprop.recipe.document.encode_integer(value)[source]#

Write an integer as TOML holds it, the inverse of decode_integer().

Parameters:

valueint: The integer.

Returns:

int or str: value itself within 64 bits, and its decimal string outside.

Examples

>>> encode_integer(2**64)
'18446744073709551616'
>>> encode_integer(42)
42

solprop.recipe.document.encode_none(value)[source]#

Write None as "none", the inverse of decode_none().

Parameters:

valueobject or None: The value.

Returns:

object: "none" for None, and otherwise value itself.

Examples

>>> encode_none(None)
'none'

solprop.recipe.document.load_array(reference, base_dir, *, allow_outside=False, path=None)[source]#

Load a numeric array from a .npy data file, with pickling disabled.

Parameters:

referenceDataFile: The reference.
base_dirPath: The recipe’s directory.
allow_outsidebool, optional: Accept a file outside base_dir (see resolve_data_file()).
pathstr or None, optional: The parameter path, for the error.

Returns:

numpy.ndarray: The array, of a boolean, integer, float or complex dtype.

Raises:

RecipeError: If read_data_file() refuses the file, it is not a .npy file, it holds objects (which only pickling could load) or its dtype is not numeric.

solprop.recipe.document.parse_text(text)[source]#

Read a recipe’s TOML text into a checked document of this format.

Parameters:

textstr: The TOML text.

Returns:

Document: The document, checked and upgraded by validate_document().

Raises:

RecipeError: If the text is not TOML, or the document is refused by validate_document().

Examples

>>> text = '''
... [recipe]
... format = 1
... route = "capillary"
...
... [build]
... radius = "125 um"
... '''
>>> parse_text(text)["build"]
{'radius': '125 um'}

solprop.recipe.document.read_data_file(reference, base_dir, *, allow_outside=False, path=None)[source]#

Read a data file’s bytes, checking its SHA-256 digest.

The bytes are read once and checked, so what the caller parses is exactly what was checked.

Parameters:

referenceDataFile: The reference.
base_dirPath: The recipe’s directory.
allow_outsidebool, optional: Accept a file outside base_dir (see resolve_data_file()).
pathstr or None, optional: The parameter path, for the error.

Returns:

bytes: The file’s contents.

Raises:

RecipeError: If resolve_data_file() refuses the file, or its digest is not the one recorded.

solprop.recipe.document.resolve_data_file(reference, base_dir, *, allow_outside=False, path=None)[source]#

Return the real path of a data file, refusing one outside the recipe’s directory.

Symbolic links are resolved first, in the directory and in the file’s path, so a link inside the directory that points out of it is refused too.

Parameters:

referenceDataFile: The reference.
base_dirPath: The recipe’s directory.
allow_outsidebool, optional: Accept a file outside base_dir; only the caller, never the recipe, sets it.
pathstr or None, optional: The parameter path, for the error.

Returns:

Path: The resolved path of an existing regular file.

Raises:

RecipeError: If the file does not exist or is not a regular file, or lies outside base_dir without allow_outside.

solprop.recipe.document.sha256_digest(data)[source]#

Return the SHA-256 digest a data-file reference records for data.

Parameters:

databytes: The file’s bytes.

Returns:

str: 64 lower-case hexadecimal digits.

Examples

>>> sha256_digest(b"")[:16]
'e3b0c44298fc1c14'

solprop.recipe.document.unknown_key(key, valid, *, where, path=None)[source]#

Return the error for an unknown key, naming the nearest valid one.

Parameters:

keystr: The key found.
validcollection of str: The keys allowed there.
wherestr: Where the key was found, such as "[execution]".
pathstr or None, optional: The parameter path, for the error.

Returns:

RecipeError: The error, to raise; it names the nearest valid key when one is close, and otherwise lists them all.

Examples

>>> print(unknown_key("field", ["fields", "field_planes"], where="[execution]"))
unknown key 'field' in [execution]; did you mean 'fields'?

solprop.recipe.document.upgrade(document)[source]#

Return a document upgraded from its format to FORMAT.

Each format after the first adds one step, from format n to n + 1, which rewrites what changed; a document is passed through the steps from its own format. Format 1 is the first, so for now this returns the document unchanged (as a new document).

Parameters:

documentMapping: A document whose [recipe] format is an integer between 1 and FORMAT.

Returns:

Document: The document in format FORMAT.

Raises:

RecipeError: If the format is not one this solprop reads.

Examples

>>> upgrade({"recipe": {"format": 1, "route": "gnlse"}})
{'recipe': {'format': 1, 'route': 'gnlse'}}

solprop.recipe.document.validate_document(document)[source]#

Check a recipe document’s tables and format, and upgrade it to this format.

Parameters:

documentMapping: The document, as tomllib reads it.

Returns:

Document: A new document in format FORMAT; the values are not copied.

Raises:

RecipeError: If a top-level name is not one of TABLES, a table is not a table, [recipe] is missing or wrong, the format is newer than FORMAT, [build] holds one of REFUSED_BUILD_KEYS, [execution] holds a key not in EXECUTION_KEYS, or a value is not a recipe value.

Examples

>>> document = validate_document(
...     {"recipe": {"format": 1, "route": "gnlse"}, "build": {"fwhm": "30 fs"}}
... )
>>> document["build"]
{'fwhm': '30 fs'}