hoops_ai.ml.metric_explorer

Classes

MetricExplorer(metric_or_flow_file_path)

Class for exploring metrics stored in a metric file.
class hoops_ai.ml.metric_explorer.MetricExplorer(metric_or_flow_file_path)

Bases: object

Class for exploring metrics stored in a metric file. It leverages an internal MetricStorage instance to load various types of metric data, and provides convenience methods for querying and post-processing those metrics.

Parameters:

metric_or_flow_file_path (str)

best_worst_categories(name, epoch)

Returns the best and worst performing categories for a given metric and epoch.

Parameters:

  • name (str) – Category metric name.

  • epoch (int) – Epoch to analyze.

Returns:

Dict with “best” and “worst” keys, each mapping to (category, value).

Return type:

Dict[str, Tuple[int, float]]

compute_confusion_matrix_stats(name, epoch)

Example of a post-processing method that, if ‘name’ corresponds to a confusion matrix, computes overall accuracy or other metrics.

Parameters:

  • name (str) – Matrix metric name (e.g., “confusion_matrix”).

  • epoch (int) – Epoch to analyze.

Returns:

A dict of computed stats, or None if matrix is invalid or not square.

Return type:

Dict[str, float] | None

get_aggregated_confusion_matrix(name, epochs)

Aggregates confusion matrices across multiple epochs by summing them.

This is useful for analyzing a “total” confusion matrix over a set of epochs rather than looking at each epoch individually.

Parameters:

  • name (str) – The matrix metric name (e.g., “confusion_matrix”).

  • epochs (List[int]) – A list of epochs to aggregate.

Returns:

A NumPy array representing the summed confusion matrix, or None if invalid.

Return type:

numpy.ndarray | None

get_category_metric(name, epoch)

Retrieves the category labels and corresponding values for a given epoch.

Parameters:

  • name (str) – Name of the category metric (e.g., “per_class_accuracy”).

  • epoch (int) – The epoch to retrieve.

Returns:

A tuple (categories, values).

Return type:

Tuple[List[int], List[float]]

get_category_over_time(name, category_id)

Retrieves the change of a specific category’s value over all epochs for a given metric.

NOTE: This relies on the fact that each category metric is stored once with multiple epochs, categories, and values. We’ll look up all epochs from the store and return the values only for the requested category.

Parameters:

  • name (str) – The category metric name (e.g., “per_class_accuracy”).

  • category_id (int) – The specific category to track.

Returns:

A tuple (epochs_list, values_for_category).

Return type:

Tuple[List[int], List[float]]

get_data(name, data_id)

Retrieves the data for a given prediction. :param name: Name of the data (e.g., “face_label_prediction”). :param data_id: The data id to retrieve. :return: A dictionary with the data fields.

Parameters:
Return type:

numpy.ndarray

get_matrix_metric(name, epoch)

Retrieves the matrix metric for a given epoch.

Parameters:

  • name (str) – The matrix metric name (e.g., “confusion_matrix”).

  • epoch (int) – The epoch to retrieve.

Returns:

A 2D NumPy array representing the stored matrix.

Return type:

numpy.ndarray

get_trend_metric(name)

Retrieves the epochs and values for a given trend metric.

Parameters:

name (str) – Name of the trend metric (e.g., “train_loss”).

Returns:

A tuple (epochs, values).

Return type:

Tuple[List[int], List[float]]

list_category_metrics()

Lists all available category metric names in the storage.

Returns:

A list of category metric names without internal structure details.

Return type:

List[str]

list_data()

Lists all available predictions in the storage. :return: A list of prediction names.

Return type:

List[str]

list_data_ids(name)

Retrieves the data ids for which predictions were made. :return: A list of data ids.

Parameters:

name (str)

Return type:

List[int]

list_matrix_metrics()

Lists all available matrix metric names in the storage.

Returns:

A list of matrix metric names without internal structure details.

Return type:

List[str]

list_trend_metrics()

Lists all available trend metric names in the storage.

Returns:

A list of trend metric names without internal structure details.

Return type:

List[str]

print_table_of_content()

Prints the table of content of the metric file, organized by metric type.

smooth_trend_metric(name, window_size=5)

Returns a smoothed version of the trend metric using a simple moving average.

Parameters:

  • name (str) – The name of the trend metric.

  • window_size (int) – The size of the moving window for smoothing.

Returns:

A tuple (epochs, smoothed_values).

Return type:

Tuple[List[int], List[float]]

summarize_trend_metric(name)

Returns a basic summary of a trend metric: min, max, mean, and last epoch value.

Parameters:

name (str) – Name of the trend metric.

Returns:

Dictionary with ‘min’, ‘max’, ‘mean’, and ‘last’ keys.

Return type:

Dict[str, int | float]