hoops_ai.storage

Quick Overview

Modules

hoops_ai.storage.datasetstorage

Classes

DataStorage()

JsonStorageHandler(json_dir_path)

Handles the storage and retrieval of data using JSON files.

OptStorage(store_path[, compress_extension])

Handles storage and retrieval of data using Zarr, including:

MemoryStorage()

MLStorage()

DGLGraphStoreHandler()

LabelStorage(path_for_storing[, ...])

Class for encoding and decoding labels.

MetricStorage(store)

Abstract class defining the interface for storing machine learning metrics based on their type of data and visualization.

CADFileRetriever(storage_provider[, ...])

LocalStorageProvider(directory_path)

Functions

convert_storage(source_handler, dest_handler)

Converts data from one storage handler to another.

Data Storage Module

The Storage module provides persistent storage solutions for CAD data, ML models, and analysis results. It offers a unified interface for various storage backends, optimized for the unique requirements of CAD data processing and machine learning workflows.

This module handles the efficient storage and retrieval of large-scale CAD datasets, encoded geometric data, trained ML models, and experimental results. It provides both high-performance options for production use and convenient formats for development and prototyping.

For storage architecture details and usage patterns, see the Data Storage Programming Guide.

class hoops_ai.storage.CADFileRetriever(storage_provider, formats=None, filter_pattern=None, use_regex=False)

Bases: object

Parameters:

storage_provider (StorageProvider)
formats (List[str] | None)
filter_pattern (str | None)
use_regex (bool)

get_file_list()

Returns the filtered list of CAD file paths.

Return type:: List[str]

class hoops_ai.storage.DataStorage

Bases: ABC

abstract close()

Handles any necessary cleanup or resource deallocation.

Return type:: None

compress_store()

Return type:: int

abstract format()

a string spcifying the concrete format for this storage.

Return type:: str

abstract get_file_path(data_key)

Retrieves the file path for a given data key.

Parameters:: data_key (str) – The data key.
Returns:: The file path corresponding to the data key.
Return type:: str

get_group_for_array(array_name)

Determines which group an array belongs to based on the schema.

Parameters:: array_name (str) – Name of the array
Returns:: Group name for the array, or None if not found in schema
Return type:: str

abstract get_keys()

Retrieves a list of all keys in the storage. :returns: A list of all keys in the storage. :rtype: list

Return type:: list

get_schema()

Retrieves the schema definition for this storage instance.

Returns:: The schema definition, or empty dict if no schema is set
Return type:: dict

abstract load_data(data_key)

Loads data associated with a specific key.

Parameters:: data_key (str) – The key of the data to load.
Returns:: The loaded data.
Return type:: Any

abstract load_metadata(key)

Loads metadata associated with a specific key.

Parameters:: key (str) – The metadata key.
Returns:: The loaded metadata value.
Return type:: Any

abstract save_data(data_key, data)

Saves data associated with a specific key.

Parameters:

data_key (str) – The key under which to store the data.
data (Any) – The data to store.

Return type:

None

abstract save_metadata(key, value)

Saves metadata as a key-value pair into the metadata JSON file. If the file doesn’t exist, it will be created.

Parameters:

key (str) – The metadata key.
value (Any) – The metadata value (bool, int, float, string, list, or array).

Return type:

None

set_schema(schema)

Sets a schema definition for this storage instance. The schema defines how arrays should be organized into groups for merging.

Parameters:: schema (dict) – Schema definition containing group and array specifications
Return type:: None

validate_data_against_schema(data_key, data)

Validates data against the stored schema if present.

Parameters:

data_key (str) – The key under which the data will be stored
data (Any) – The data to validate

Returns:

True if valid or no schema present, False if validation fails

Return type:

bool

class hoops_ai.storage.JsonStorageHandler(json_dir_path)

Bases: DataStorage

Handles the storage and retrieval of data using JSON files.

Each data_key is stored as a separate JSON file within a specified directory.

Parameters:: json_dir_path (str)

close()

Closes the JsonStorageHandler. Since JSON operations are stateless, there’s nothing to close.

Return type:: None

compress_store()

Return type:: int

format()

a string specifying the concrete format for this storage.

Return type:: str

get_file_path(data_key)

Retrieves the file path for a given data key.

Parameters:: data_key (str) – The data key.
Returns:: The file path corresponding to the data key.
Return type:: str

get_group_for_array(array_name)

Determines which group an array belongs to based on the schema.

Parameters:: array_name (str) – Name of the array
Returns:: Group name for the array, or None if not found in schema
Return type:: str

get_keys()

Lists all data keys available in the JSON storage directory.

Returns:: A list of data keys.
Return type:: list

get_schema()

Retrieves the schema definition for this storage instance.

Returns:: The schema definition, or empty dict if no schema is set
Return type:: dict

load_data(data_key)

Loads data from a JSON file using the specified key.

Parameters:

data_key (str) – The key of the data to load.

Returns:

The loaded data.

Return type:

Any

Raises:

KeyError – If the data_key does not exist.
ValueError – If the JSON data cannot be deserialized.

load_metadata(key)

Loads metadata by key from the metadata JSON file. Supports nested keys using ‘/’ as a separator.

Parameters:: key (str) – The metadata key, which can be a nested key using ‘/’ as a separator.
Returns:: The loaded metadata value.
Return type:: Any
Raises:: KeyError – If the key does not exist.

save_data(data_key, data)

Saves data to a JSON file under the specified key.

Parameters:

data_key (str) – The key under which to store the data.
data (Any) – The data to store.

Raises:

ValueError – If the data cannot be serialized to JSON.

Return type:

None

save_metadata(key, value)

Saves metadata as a key-value pair into the metadata JSON file. If the file doesn’t exist, it will be created.

Parameters:

key (str) – The metadata key, which can be a nested key using ‘/’ as a separator.
value (Any) – The metadata value (bool, int, float, string, list, or array).

Return type:

None

set_schema(schema)

Sets a schema definition for this storage instance. The schema defines how arrays should be organized into groups for merging.

Parameters:: schema (dict) – Schema definition containing group and array specifications
Return type:: None

validate_data_against_schema(data_key, data)

Validates data against the stored schema if present.

Parameters:

data_key (str) – The key under which the data will be stored
data (Any) – The data to validate

Returns:

True if valid or no schema present, False if validation fails

Return type:

bool

class hoops_ai.storage.LocalStorageProvider(directory_path)

Bases: StorageProvider

Parameters:: directory_path (str)

list_files(extensions)

Retrieves CAD file paths from a local directory or a text file using pathlib.

Parameters:: extensions (List[str])
Return type:: List[str]

hoops_ai.storage.convert_storage(source_handler, dest_handler)

Converts data from one storage handler to another.

Parameters:

source_handler (DataStorageHandlerBase) – The source storage handler (e.g., JSON handler).
dest_handler (DataStorageHandlerBase) – The destination storage handler (e.g., Zarr handler).

Return type:

None

hoops_ai.storage

Data Storage Module

Hello! I'm HOOPSY