hoops_ai.storage.datastorage.zarr_storage_handler

Classes

OptStorage(store_path[, compress_extension])

Handles storage and retrieval of data using Zarr, including:
class hoops_ai.storage.datastorage.zarr_storage_handler.OptStorage(store_path, compress_extension='.data')

Bases: DataStorage

Handles storage and retrieval of data using Zarr, including:

  • Setting dimension names so that xarray can open each array.

  • Checking for NaNs before writing and before zipping.

  • Compressing into a .zip-based Zarr store.

Parameters:

  • store_path (str)

  • compress_extension (str)

close()

Deletes the entire DirectoryStore if needed. Thread-safe.

Return type:

None

compress_store()

Copies data from the DirectoryStore to a ZipStore, verifying no NaNs exist in raw arrays. If everything is OK, removes the DirectoryStore.

Return type:

int

format()

Returns ‘zarr’ to match your original interface.

Return type:

str

get_file_path(data_key)

Returns the direct path for a particular data key (sub-directory or array).

Parameters:

data_key (str)

Return type:

str

get_group_for_array(array_name)

Determines which group an array belongs to based on the schema.

Parameters:

array_name (str) – Name of the array

Returns:

Group name for the array, or None if not found in schema

Return type:

str

get_keys()

Retrieves a list of all top-level keys in the storage. :returns: A list of top-level keys in the storage. :rtype: list

Return type:

list

get_schema()

Retrieves the schema definition for this storage instance.

Returns:

The schema definition, or empty dict if no schema is set

Return type:

dict

load_data(data_key)

Loads data from the Zarr store by key.

Parameters:

data_key (str)

Return type:

Any

load_metadata(key)

Loads custom metadata by key from the local metadata file.

Parameters:

key (str)

Return type:

Any

save_data(data_key, data)

Saves data (numpy arrays, lists, dicts, scalars, strings) under data_key. Overwrites if that key already exists.

Parameters:
Return type:

None

save_metadata(key, value)

Saves custom metadata (JSON-serializable) to the local metadata file.

Parameters:
Return type:

None

set_schema(schema)

Sets a schema definition for this storage instance. The schema defines how arrays should be organized into groups for merging.

Parameters:

schema (dict) – Schema definition containing group and array specifications

Return type:

None

validate_data_against_schema(data_key, data)

Validates data against the stored schema if present.

Parameters:

  • data_key (str) – The key under which the data will be stored

  • data (Any) – The data to validate

Returns:

True if valid or no schema present, False if validation fails

Return type:

bool