Loader

Theses classes are responsible for data loading and manage the process of reading raw input data, applying basic preprocessing steps, and converting it into the internal format used by the framework.

Load a single dataset

scentree.io.loader.Dataset

Bases: BaseModel

Container for the minimal information required to describe a dataset.

Attributes:

Name	Type	Description
name	str	The name of the dataset.
values	NDArray[float64]	The matrix containing the data.
stage_ids	List[int]	The stage each column belongs to.
bounds	Bounds	Optional tuple defining the lower and upper bounds of values.

bounds class-attribute instance-attribute

bounds: Bounds = None

model_config class-attribute instance-attribute

model_config = {'arbitrary_types_allowed': True}

name instance-attribute

name: str

stage_ids instance-attribute

stage_ids: List[int]

values instance-attribute

values: NDArray[float64]

validate_consistency

validate_consistency() -> Self

Validate the internal consistency of the dataset specification.

This validator ensures that the metadata describing the dataset (stage_ids and bounds) is consistent with the structure of values.

The following conditions must hold

• The length of stage_ids must match the number of columns in values.
• If bounds is not sorted, i.e., first value must be less than second value.

Returns:

Name	Type	Description
Self	Self	The validated instance.

Raises:

Type	Description
ValueError	If any of the consistency conditions are violated.

Load multiple datasets

scentree.io.loader.DatasetsLoader

Bases: BaseModel

Container that represents a collection of datasets.

Attributes:

Name	Type	Description
datasets	List[Dataset]	Collection of datasets.

datasets instance-attribute

datasets: List[Dataset]

model_config class-attribute instance-attribute

model_config = {'arbitrary_types_allowed': True}

create_stages_columns_mapping

create_stages_columns_mapping() -> DatasetMappings

Create the mapping between datasets, stages, and column positions.

This method constructs a mapping describing how the columns of the full stage-ordered matrix are associated with each dataset and stage. Column indices are assigned according to the ordering produced by get_stage_values.

Returns:

Name	Type	Description
DatasetMappings	DatasetMappings	List of dataset mappings. Each mapping contains: - dataset: dataset name - columns: column indices associated with the dataset - stage_ids: stages in which the dataset appears

get_full_bounds

get_full_bounds() -> FullBouds

Flatten the stage-wise bounds structure into a single list.

Returns:

Name	Type	Description
FullBouds	FullBouds	Flattened list containing the bounds associated with all variables across all stages. Returns None if no bounds are defined.

get_full_values

get_full_values() -> NDArray[np.float64]

Construct the full data matrix by concatenating all stage matrices.

Returns:

Type	Description
NDArray[float64]	NDArray[np.float64]: Full data matrix containing all variables across all stages. Rows correspond to observations and columns correspond to stage-ordered variables.

get_num_variables_per_stage

get_num_variables_per_stage() -> List[int]

Compute the number of variables associated with each stage.

Returns:

Type	Description
List[int]	List[int]: List containing the number of variables for each stage. Each position corresponds to a stage in sorted order.

get_sorted_stage_ids

get_sorted_stage_ids() -> List[int]

Retrieve all unique stage identifiers sorted in ascending order.

Returns:

Type	Description
List[int]	List[int]: Sorted list of unique stage identifiers across all datasets.

get_stage_bounds

get_stage_bounds() -> StageBounds

Construct the bounds associated with each stage.

Returns:

Name	Type	Description
StageBounds	StageBounds	Nested list containing the bounds associated with each variable of each stage. The outer list corresponds to stages, while the inner lists correspond to variables within the stage. Returns None if no dataset defines bounds.

get_stage_values

get_stage_values() -> List[NDArray[np.float64]]

Construct the data matrices associated with each stage.

Returns:

Type	Description
List[NDArray[float64]]	List[NDArray[np.float64]]: List of stage matrices ordered according to the sorted stage identifiers. Each matrix contains all variables associated with the corresponding stage.

validate_datasets_values

validate_datasets_values() -> Self

Validate that all datasets contain the same number of rows.

This validator checks that the values matrix of every dataset in datasets_information has the same number of rows. The number of rows represents the number of observations, which must be consistent across all datasets.

Raises:

Type	Description
ValueError	If any dataset contains a different number of rows than the other datasets.

Returns:

Name	Type	Description
Self	Self	The validated model instance.