hoops_ai.storage.metric_storage

Classes

MetricStorage(store)

Abstract class defining the interface for storing machine learning metrics based on their type of data and visualization.
class hoops_ai.storage.metric_storage.MetricStorage(store)

Bases: object

Abstract class defining the interface for storing machine learning metrics based on their type of data and visualization.

Parameters:

store (DataStorage)

get_storage()

Returns the storage handler for this metric storage.

Returns:

The storage handler instance.

list_data_ids(name)

Returns all data_ids pushed under ‘name’ by examining the Zarr store keys.

Parameters:

name (str)

Return type:

List[int]

pull_category_metric(name, epoch)

Pulls category-based metric data for a specific epoch from storage.

Parameters:

  • name (str) – The metric name (e.g., “per_class_accuracy”).

  • epoch (int) – The epoch for which to retrieve the metric.

Returns:

A tuple containing a list of categories and the corresponding metric values.

Raises:

ValueError – If the specified epoch is not found.

Return type:

Tuple[List[int], List[float]]

pull_data(name, data_id)

Loads prediction data from storage.

Parameters:

  • name (str) – The name of the prediction metric.

  • data_id (int) – The identifier for the data.

Returns:

A NumPy array containing the prediction results.

Return type:

numpy.ndarray

pull_matrix_metric(name, epoch)

Pulls a matrix-based metric for a specific epoch from storage.

Parameters:

  • name (str) – The metric name (e.g., “confusion_matrix”, “feature_correlation”).

  • epoch (int) – The epoch for which to retrieve the matrix metric.

Returns:

A 2D NumPy array representing the matrix metric.

Raises:

ValueError – If the specified epoch is not found.

Return type:

numpy.ndarray

pull_trend_metric(name)

Pulls trend metric data from storage.

Parameters:

name (str) – The metric name (e.g., “train_loss”, “val_accuracy”).

Returns:

A tuple containing a list of epochs and corresponding metric values.

Return type:

Tuple[List[int], List[float]]

push_category_metric(name, epoch, categories, values)

Pushes category-based metrics incrementally in memory before writing them to storage.

Reason: Compares performance across different categories (e.g., classes, features) for each epoch.

Metrics Included: - Per-Class Accuracy (one bar per class) - Per-Class IoU (one bar per class) - Feature Importance (e.g., in random forests, SHAP values)

Parameters:

  • name (str) – The metric name (e.g., “per_class_accuracy”).

  • epoch (int) – The current epoch index.

  • categories (List[int]) – List of category labels (e.g., class indices, feature indices).

  • values (List[float] | List[int]) – List of metric values corresponding to each category.

push_matrix_metric(name, epoch, matrix)

Pushes matrix-based metrics incrementally in memory before writing them to storage.

Reason: Stores structured relationships between multiple variables in matrix form, per epoch.

Metrics Included: - Confusion Matrix (classification tasks) - Correlation Matrix (e.g., feature correlations)

Parameters:

  • name (str) – The metric name (e.g., “confusion_matrix”, “feature_correlation”).

  • epoch (int) – The current epoch index.

  • matrix (numpy.ndarray) – A 2D NumPy array representing the metric (e.g., confusion matrix).

push_predictions(name, data_id, result)

Saves prediction data to storage.

Parameters:

  • name (str) – The name of the prediction metric.

  • data_id (int) – The identifier for the data.

  • result (numpy.ndarray) – A NumPy array containing the prediction results.

push_trend_metric(name, epoch, value)

Pushes trend metrics incrementally in memory before writing to a file.

Reason: Tracks values over time (epochs) to analyze learning progress.

Metrics Included: - Loss (training, validation, test) - Accuracy over epochs - Precision/Recall over epochs - F1-score over epochs - IoU (mean IoU, per-class IoU over time) - RMSE/MSE/MAE for regression tasks over epochs - Learning Rate schedules

Parameters:

  • name (str) – The metric name (e.g., “train_loss”, “val_accuracy”).

  • epoch (int) – The current epoch index.

  • value (float) – The metric value at the given epoch.

store(compress=False)

Writes all stored metrics in memory to the storage handler and clears memory storage.

Parameters:

compress (bool) – If True, compresses the store after saving the data.