flambe.learn.eval

Module Contents

class flambe.learn.eval.Evaluator(dataset: Dataset, eval_sampler: Sampler, model: Module, metric_fn: Metric, eval_data: str = 'test', device: Optional[str] = None)[source]

Bases: flambe.compile.Component

Implement an Evaluator block.

An Evaluator takes as input data, and a model and executes the evaluation. This is a single step Component object.

Parameters:
  • dataset (Dataset) – The dataset to run evaluation on
  • eval_sampler (Sampler) – The sampler to use over validation examples
  • model (Module) – The model to train
  • metric_fn (Metric) – The metric to use for evaluation
  • eval_data (str) – The data split to evaluate on: one of train, val or test
  • device (str, optional) – The device to use in the computation.
run(self, block_name: str = None)[source]

Run the evaluation.

Returns:Whether the computable has completed.
Return type:bool
metric(self)[source]

Override this method to enable scheduling.

Returns:The metric to compare computable varients
Return type:float