flambe.metric.metric

Module Contents

class flambe.metric.metric.Metric[source]

Bases: flambe.compile.Component

Base Metric interface.

Objects implementing this interface should take in a sequence of examples and provide as output a processd list of the same size.

compute(self, pred: torch.Tensor, target: torch.Tensor)[source]

Computes the metric over the given prediction and target.

Parameters:
  • pred (torch.Tensor) – The model predictions
  • target (torch.Tensor) – The ground truth targets
Returns:

The computed metric

Return type:

torch.Tensor

aggregate(self, state: dict, *args, **kwargs)[source]

Aggregates by simply storing preds and targets

Parameters:
  • state (dict) – the metric state
  • args (the pred, target tuple) –
Returns:

the state dict

Return type:

dict

finalize(self, state: Dict)[source]

Finalizes the metric computation

Parameters:state (dict) – the metric state
Returns:The final score.
Return type:float
__call__(self, *args, **kwargs)[source]

Makes Featurizer a callable.

__str__(self)[source]

Return the name of the Metric (for use in logging).

class flambe.metric.metric.AverageableMetric[source]

Bases: flambe.metric.metric.Metric

Metric interface for averageable metrics

Some metrics, such as accuracy, are averaged as a final step. This allows for a more efficient metrics computation. These metrics should inherit from this class.

aggregate(self, state: dict, *args, **kwargs)[source]
Parameters:
  • state (dict) – the state dictionary
  • args – normally pred, target
  • kwargs
Returns:

The updated state (even though the update happens in-place)

Return type:

dict

finalize(self, state)[source]

FInalizes the metric computation

Parameters:state (dict) – the metric state
Returns:The final score. Can be anything, depending on metric.
Return type:Any