flambe.experiment.utils

Module Contents

Check validity of links between blocks.

Ensures dependency order, and that only Comparable blocks are being reduced through a LinkBest object.

Parameters:

blocks (OrderedDict[str, Schema[Component]]) – The blocks to check, in order

Raises:
  • LinkError – On undeclared blocks (i.e not the right config order)
  • ProtocolError – Attempt to reduce a non-comparable block

Check validity of links between blocks.

Ensures dependency order, and that only Comparable blocks are being reduced through a LinkBest object.

Parameters:
  • blocks (OrderedDict[str, Schema[Component]]) – Ordered mapping from block id to a schema of the block
  • search (Mapping[str, SearchAlgorithm], optional) – Map from block id to hyperparameter search space generator
  • schedulers (Mapping[str, TrialScheduler], optional) – Map from block id to search scheduler
Raises:
  • ProtocolError – Non computable block assigned a search or scheduler.
  • ProtocolError – Non comparable block assigned a non default search or scheduler
flambe.experiment.utils.convert_tune(data: Any)[source]

Convert the options and links in the block.

Convert Option objects to tune.grid_search or tune.sample_from functions, depending on the type.

Parameters:data (Any) – Input object that may contain Options objects that should be converted to a Tune-compatible representation
flambe.experiment.utils.traverse(nested: Mapping[str, Any], path: Optional[List[str]] = None) → Iterable[Any][source]

Iterate over a nested mapping returning the path and key, value.

Parameters:
  • nested (Mapping[str, Any]) – Mapping where some values are also mappings that should be traversed
  • path (List[str]) – List of keys that were used to reach the current mapping
Returns:

Iterable of path, key, value triples

Return type:

Iterable[Any]

flambe.experiment.utils.traverse_all(obj: Any) → Iterable[Any][source]

Iterate over all nested mappings and sequences

Parameters:obj (Any) –
Returns:Iterable of child values to obj
Return type:Iterable[Any]
flambe.experiment.utils.traverse_spec(nested: Mapping[str, Any], path: Optional[List[str]] = None) → Iterable[Any][source]

Iterate over a nested mapping returning the path and key, value.

Parameters:
  • nested (Mapping[str, Any]) – Mapping where some values are also mappings that should be traversed
  • path (List[str]) – List of keys that were used to reach the current mapping
Returns:

Iterable of path, key, value triples

Return type:

Iterable[Any]

flambe.experiment.utils.update_nested(nested: MutableMapping[str, Any], prefix: Iterable[str], key: str, new_value: Any) → None[source]

Multi-level set operation for nested mapping.

Parameters:
  • nested (Mapping[str, Any]) – Nested dictionary where keys are all strings
  • prefix (Iterable[str]) – List of keys specifying path to value to be updated
  • key (str) – Final key corresponding to value to be updated
  • new_value (Any) – New value to set for [p1]…[key] in nested
flambe.experiment.utils.get_nested(nested: Mapping[str, Any], prefix: Iterable[str], key: str) → Any[source]

Get nested value in standard Mapping.

Parameters:
  • nested (Mapping[str, Any]) – The mapping to index in
  • prefix (Iterable[str]) – The path to the final key in the nested input
  • key (str) – The key to query
Returns:

The value at the given path and key

Return type:

Any

flambe.experiment.utils.update_schema_with_params(schema: Schema, params: Dict[str, Any]) → Schema[source]

Replace options in the schema recursivly.

Parameters:
  • schema (Schema[Any]) – The schema object to update
  • params (Dict[str, Any]) – The corresponding nested diciontary with values
Returns:

The update schema (same object as the input, not a copy)

Return type:

Schema[Any]

flambe.experiment.utils.has_schemas_or_options(x: Any) → bool[source]

Check if object contains Schemas or Options.

Recurses for Mappings and Sequences

Parameters:x (Any) – Input object to check for Schemas and Options
Returns:True iff contains any Options or Schemas.
Return type:bool
flambe.experiment.utils.divide_nested_grid_search_options(config: MutableMapping[str, Any]) → Iterable[Mapping[str, Any]][source]

Divide config into a config Iterable to remove nested Options.

For every GridSearchOptions or SampledUniformSearchOptions, if any values contain more Options or Schemas, create copies with a single value selected in place of the option. Resulting configs will have no nested options.

Parameters:config (MutableMapping[str, Any]) – MutableMapping (or Schema) containing Options and Schemas
Returns:Each Mapping contains variants from original config without nested options
Return type:Iterable[Mapping[str, Any]]
flambe.experiment.utils.extract_dict(config: Mapping[str, Any]) → Dict[str, Any][source]

Turn the schema into a dictionary, ignoring types.

NOTE: We recurse if any value is itself a Schema, a Sequence of Schema`s, or a `Mapping of `Schema`s. Other unconvential collections will not be inspected.

Parameters:schema (Schema) – The object to be converted into a dictionary
Returns:The output dictionary representation.
Return type:Dict
flambe.experiment.utils.extract_needed_blocks(schemas: Dict[str, Schema], block_id: str, global_vars: Optional[Dict[str, Any]] = None) → Set[str][source]

Returns the set of all blocks that the input block links to.

Parameters:
  • schemas (Dict[str, Schema[Any]]) – Map from block_id to Schema object
  • block_id (str) – The block containing links
Returns:

The list of ancestor block ids

Return type:

List[str]

Resolve links in schemas at block_id.

Parameters:
  • schemas (Dict[str, Schema[Any]]) – Map from block_id to Schema object
  • block_id (str) – The block where links should be activated
  • global_vars (Dict[str, Any]) – The environment links (ex: resources)
flambe.experiment.utils.get_best_trials(trials: List[Trial], topk: int, metric='episode_reward_mean') → List[Trial][source]

Get the trials with the best result.

Parameters:
  • trials (List[ray.tune.Trial]) – The list of trials to examine
  • topk (int) – The number of trials to reduce to
  • metric (str, optional) – The metric used in comparaison (higher is better)
Returns:

The list of best trials

Return type:

List[ray.tune.Trial]

flambe.experiment.utils.get_non_remote_config(experiment)[source]

Returns a copy of the original config file without the remote configuration

Parameters:experiment (Experiment) – The experiment object
flambe.experiment.utils.local_has_gpu() → bool[source]

Returns is local process has GPU

Returns:
Return type:bool
flambe.experiment.utils.rel_to_abs_paths(d: Dict[str, str]) → Dict[str, str][source]

Convert relative paths to absolute paths.

Parameters:d (Dict[str, str]) – A dict from name -> path.
Returns:The same dict received as parameter with relative paths replaced with absolute.
Return type:Dict[str, str]
flambe.experiment.utils.shutdown_ray_node() → int[source]

Call ‘ray stop’ locally to terminate the ray node.

flambe.experiment.utils.shutdown_remote_ray_node(host: str, user: str, key: str) → int[source]

Execute ‘ray stop’ on a remote machine through ssh to terminate the ray node.

IMPORTANT: this method is intended to be run in the cluster.

Parameters:
  • host (str) – The Orchestrator’s IP that is visible by the factories (usually the private IP)
  • user (str) – The username for that machine.
  • key (str) – The key that communicate with the machine.
flambe.experiment.utils.get_default_devices(debug: bool = False, default_cpus: int = 1, default_gpus: int = 1) → Dict[str, int][source]

Get the default devices to use if not provided.

Parameters:
  • debug (bool, optional) – If we are running in debug mode (where Ray is not available). If debug is False, this method should be called after running ray.init().
  • default_cpus (int, optional) – The default number of CPU’s to use. Default 1.
  • default_gpus (int, optional) – The default number of GPU’s to use. Default 1.
Returns:

devices – Default set of devices to use. Should have at most two keys: ‘cpu’, and ‘gpu’ if cuda is available.

Return type:

Dict[str, int]

Raises:

ValueError – If the total number of GPUs is larger than the number available.