Object Construction and Property Validation for Files of Synthesis (FOS)

Introduction

During construction of a parsed FOS into a Synthesis object, or when reassigning a property of an exising object, most classes have properties that should be expected in a certain format, or might contain nested objects with their own properties. The parsed properties are passed as either strings or (in the case of nested objects) a list or dict structure to the validation routine mapped to them in FoSpy.parsing.validation.py. In the case of nested objects, the validation routine is usually the constructor of the required object.

Classes can have required properties, which must be present at read time, or optional properties, which have validation routines but might not be expected for all syntheses.

During attribute assignment, expected properties are built up in order of parent classes. Any subclass inherits all the expected properties of its parent classes, but they may be overwritten to other validators.

Dispatching Subclasses

Properties marked with (dispatched) are used to identify and delegate construction to the respective subclass. A class with a dispatch key is rarely constructed as itself, but may be used to inherit methods or required/optional properties.

`ListBlock` and Simple Lists

ListBlock subclasses do not have required properties. Instead, they are a rich list of objects with a certain required SingleBlock subclass. Some ListBlock subclasses have their own methods and unique attributes. If a ListBlock is required for a given SingleBlock subclass but it doesn't need any method or attribute overrides, it can be instantiated with the class method ListBlock.Simple(SingleBlockSubclass) (referred to below as a "simple list").

`TemplateList`s

TemplateList is a unique subclass of ListBlock used for storing TemplateBlock subclass objects. TemplateList has a class method Simple() similar to ListBlock's method, but instead of full objects (e.g. Material), it generates hybridized templates with fields for any field-marked or missing properties. Some syntax for TemplateList entries can be found in the code example walkthrough.

Modifying Property Validation at Runtime

The intent of the FOS is to be as flexible as possible while still enforcing standards for fully capturing a synthetic method. However, it may be necessary for private or niche applications to modify standards to match your own synthesis. This can be done by mutating the dictionaries in FoSpy.parsing.validation at the start of your script. The example below creates a new block type with its own expected properties, and then adds it as an optional top-level block for a synthesis file.

These modifications should be reserved for very unique and isolated cases. If you are modifying standards in a significant way, or such that may be applicable for other researchers, consider reaching out to FoSpy developers or creating a fork of the GitHub.

Python

import FoSpy.parsing.validation as vd
from FoSpy.blocks import (
    blocks.SingleBlock as SingleBlock
    synthesis.Synthesis as Synthesis
)


class MySpecialBlock(SingleBlock):
    pass

vd.required_keys[MySpecialBlock] = {
    "special_prop1": str
    "special_prop2": float
}

vd.optional_keys[Synthesis]["special"] = MySpecialBlock

New FOS Block

[Special]
special_prop1: foobar
special_prop2: 6.022

Property	Validation Routine or Class	Description
program	`AnnealProgram`	A specialized `ListBlock` of `AnnealSection` objects
start_temp	`validators.numbers.positive_decimal`	The initial temperature of the annealing profile. Values are attached to the required unit and constructed into a `pint.Quantity` object.
start_temp_unit	`validators.units.FOSTempUnit`	`FOSTempUnit` is a subclass of `pint`'s `Unit` which allows a little more coersion of temperature units. (Like recognizing `"C"` as degrees celsius as opposed to coulombs)

Property	Validation Routine or Class	Description
file_name	`validators.filenames.file_name`	A name for the file that does not contain any incompatible characters ( `\ / : * ? " < > \\|`). Extensions are allowed, but will be trimmed if matching extension property, or appended to if not matching.
extension	`validators.filenames.file_extension` (dispatched)	A valid file extension (with or without "`.`" prefix).

Property	Validation Routine or Class	Description
embedded	`list`	A list of raw utf-8 line strings copied from the embedded file. See attachments guide for syntax.
path	`validators.filenames.PathPosix`	Instead of directly embedding contents, refers to a relative path from the folder containing the parent `FileBlock`. Can use relative characters like "`.`" and "`..`". See attachments guide for more information. `PathPosix` validator is a subclass of `pathlib.Path` which always uses back slashes (`/`) instead of forward slashes (`\`) when serialized, regardless of OS

Property	Validation Routine or Class	Description
time	`validators.numbers.positive_decimal`	How long the temperature was kept constant in this section. Values are attached to the required unit and constructed into a `pint.Quantity` object.
time_unit	`validators.units.FOSUnit.enforce_dims('[time]')`	`FOSUnit` is a subclass of `pint`'s `Unit` with a class method for enforcing the correct dimensionality of the unit.

Property	Validation Routine or Class	Description
name	`str`	Name of the experimenter
affiliation	`str`	Lab/University/Research Group/etc.

Property	Validation Routine or Class	Description
name	`str`	Identifiable name for the material
type	`str`	How it was used in the synthesis (e.g., reagent, flux, solvent)
formula	`ChemFormula`	Chemical composition written in a ChemFormula compatible format
supplier	`str`	Source of purchase/synthesis (may be internal)
cas	`str`	CAS ID. May be "unknown"
form	`str`	Physical shape or state of the material at time of acquisition (e.g., powder, shot, wire, lump). If the material was modified after aquiring but before use in the synthesis (like grinding into powder, drying, etc.), these actions should be specified in the material's treatments property (not the synthesis treatments).
env	`str`	What environment the material is stored in. (e.g., ambient, Ar(g))
amount	`validators.numbers.positive_decimal`	Amount that was used. Values are attached to the required unit and constructed into a `pint.Quantity` object.
amount_unit	`str`	Descriptive unit for amount. Dimensionality may be enforced in the future once more input is gained from experimenters.

Property	Validation Routine or Class	Description
purity	`validators.numbers.decimal_range`	0 < purity \<= 1
treatments	`TreatmentList`	A simple list of `Treatment` objects Any modifications to the material between acquisition and use in the synthesis.

Property	Validation Routine or Class	Description
fos_id	`str`	Identifiable ID for the synthesis or sample. For `Synthesis` files: this should be kept informative and unique within the scope of the project_id, as it may be used as identification for future database storage. For `TemplateSet` files: it is for convenience.
fos_type	`str`	What type of `FileBlock` subclass the file should be interpreted as. Expected values are: `synthesis` `templates`
description	`str`	A brief description of the intent for the file (characteristic methods, target products, template category, etc.).

Property	Validation Routine or Class	Description
name	`str`	Identifiable name for the product.
expected	`bool`	Whether or not it was expected from the synthesis.
obtained	`bool`	Whether or not it was obtained from the synthesis.
formula	`ChemFormula`	Chemical composition written in a ChemFormula compatible format.
observations	`str`	General observations about the product.

Property	Validation Routine or Class	Description
expected_amount	`validators.numbers.positive_decimal`	How much of the product was nominally expected to be obtained from the synthesis. Values are attached to the required unit and constructed into a `pint.Quantity` object.
expected_amount_unit	`validators.units.FOSUnit.enforce_dims(["[mass]",{"[length]":3}])`	`FOSUnit` is a subclass of `pint`'s `Unit` with a class method for enforcing the correct dimensionality of the unit. (In this case mass or volume)
obtained_amount	`validators.numbers.positive_decimal`	How much of the product was actually obtained from the synthesis. Values are attached to the required unit and constructed into a `pint.Quantity` object.
obtained_amount_unit	`validators.units.FOSUnit.enforce_dims(["[mass]",{"[length]":3}])`	`FOSUnit` is a subclass of `pint`'s `Unit` with a class method for enforcing the correct dimensionality of the unit. (In this case mass or volume)
characterizations	`str`	Description of characterization methods used to determine/quantitate the product.
structure_comments	`str`	General description on the structure of the product.

Property	Validation Routine or Class	Description
cif	`Attachment.enforce_subtype(CIFFile)`	A single attached CIF file
cifs	`CifList`	A simple list of attached CIF files
laboratory_conditions	`LabConditions`	General Laboratory Conditions
equipment	`EquipmentList`	A simple list of `Equipment` objects

Property	Validation Routine or Class	Description
group_id	`str`	A unique identifier for the research group or organization. In the future: unique group_id values should be standardized and/or issued by a central party to ensure that all future database uploads have a unique index location. For now: group_id values can be verified unique by ending with the primary investigator's ORCID (example: kovnir-0000-0003-1152-1912). After standardization, tools will be developed to convert group_id values for entire group repositories.
project_id	`str`	An identifier that only needs to be unique within the scope of the group_id. This can be more flexible to the needs of the group, but good practice is to keep synthesis files in a folder structure that matches project_id values. A large experimental group, for example, might categorize syntheses by experimenter then project, or vice versa for intra-collaboration. To avoid future conflicts, name-based categories should be given unique suffixes, like university ID/username or the last 4 digits of the ORCID. Some examples: `travis5672/clathrates`, `travis(errthumt)/pnictides`, `thermoelectrics/travis5672/Ba2-TM5-Pn6` Future Tools will expect project_ids to reflect folder structure using "`/`" or "`\`" delimiters.

Property	Validation Routine or Class	Description
experimenters	`TemplateList.Simple(Experimenter)`	A list of incomplete `Experimenter` objects
materials	`TemplateList.Simple(Material)`	A list of incomplete `Material` objects
treatments	`TemplateList.Simple(Treatment)`	A list of incomplete `Treatment` objects
annealings	`TemplateList.Simple(Annealing)`	A list of incomplete `Annealing` objects
anneal_sections	`TemplateList.Simple(AnnealSection)`	A list of incomplete `AnnealSection` objects
cifs	`CifList`	A simple list of attached CIF files

Property	Validation Routine or Class	Description
temp	`validators.numbers.positive_decimal`	The next temperature in the program. Values are attached to the required unit and constructed into a `pint.Quantity` object.
time	`validators.numbers.positive_decimal`	How long it took to get from the last temperature to the new temperature. Values are attached to the required unit and constructed into a `pint.Quantity` object.
rate	`validators.numbers.any_decimal`	The sign-sensitive rate at which temperature was changed to get to the new temperature. (Increase -> positive, Decrease -> negative). Values are attached to the required unit and constructed into a `pint.Quantity` object.
temp_unit	`validators.units.FOSTempUnit`	`FOSTempUnit` is a subclass of `pint`'s `Unit` which allows a little more coersion of temperature units. (Like recognizing `"C"` as degrees celsius as opposed to coulombs)
time_unit	`validators.units.FOSUnit.enforce_dims("[time]")`	`FOSUnit` is a subclass of `pint`'s `Unit` with a class method for enforcing the correct dimensionality of the unit.
rate_unit	`validators.units.temp_rate_unit`	Makes use of `pint`'s dimensionality properties to verify that the value is a unit of temperature over time.

Property	Validation Routine or Class	Description
nominal_formula	`ChemFormula`	Chemical composition written in a ChemFormula compatible format
nominal_amount	`validators.numbers.positive_decimal`	Total amount expected to be recovered from all participating reactants. Values are attached to the required unit and constructed into a `pint.Quantity` object.
nominal_amount_unit	`validators.units.FOSUnit.enforce_dims(["[mass]",{"[length]":3}])`	`FOSUnit` is a subclass of `pint`'s `Unit` with a class method for enforcing the correct dimensionality of the unit.

Property	Validation Routine or Class	Description
metadata	`SynthesisMeta`	General information about the file. Additional requirements from basic file metadata. Lines at the beginning of a FOS-formatted file without a header will automatically be interpreted as a `MetaData` dictionary
experimenters	`ExperimenterList`	A simple list of `Experimenter` objects
reaction	`Reaction`	General reaction information
products	`ProductList`	A simple list of `Product` objects
materials	`MaterialList`	A specialized `ListBlock` of `Material` objects
treatments	`TreatmentList`	A simple list of `Treatment` objects

Property	Validation Routine or Class	Description
type	`str` (dispatched)	What type of treatment was performed
repeats	`int`	How many times it was performed in succession (if interrupted by other treatments, add a different treatment block after the interrupting treatments)
observations	`str`	General observations

Property	Validation Routine or Class	Description
recovered_amount	`validators.numbers.positive_decimal`	How much material was recovered after treatment. Values are attached to the required unit and constructed into a `pint.Quantity` object.
recovered_amount_unit	`validators.units.FOSUnit.enforce_dims(["[mass]",{"[length]":3}])`	`FOSUnit` is a subclass of `pint`'s `Unit` with a class method for enforcing the correct dimensionality of the unit.
start_time	`str`	What time the treatment was started
end_time	`str`	What time the treatment was finished

Object Construction and Property Validation for Files of Synthesis (FOS)

Introduction

Dispatching Subclasses

ListBlock and Simple Lists

TemplateLists

Modifying Property Validation at Runtime

Python

New FOS Block

Expected Property Tables

AnnealSection

Required properties

Annealing

Required properties

Optional properties

Attachment

Required properties

Additional Requirements

Optional properties

Attachment Method Subclasses

Attachment Types

File Types

Dwell

Required properties

Experimenter

Required properties

Optional properties

FileBlock

Required properties

Material

Required properties

Optional properties

MetaData

Required properties

Product

Required properties

Optional properties

Quench

Required properties

Ramp

Required properties

Optional properties

Ramp Method Subclasses

Reaction

Required properties

SingleBlock

Required properties

Optional properties

SingleBlock Method Subclasses

Synthesis

Required properties

Optional properties

SynthesisMeta

Required properties

TemplateBlock

Required properties

TemplateBlock Method Subclasses

TemplateSet

Optional properties

Treatment

Required properties

Optional properties

`ListBlock` and Simple Lists

`TemplateList`s

`AnnealSection`

`Annealing`

`Attachment`

`Dwell`

`Experimenter`

`FileBlock`

`Material`

`MetaData`

`Product`

`Quench`

`Ramp`

`Reaction`

`SingleBlock`

`Synthesis`

`SynthesisMeta`

`TemplateBlock`

`TemplateSet`

`Treatment`