flambe.runnable.cluster_runnable

Module Contents

class flambe.runnable.cluster_runnable.ClusterRunnable(user_provider: Callable[[], str] = None, env: Optional[RemoteEnvironment] = None, **kwargs)[source]

Bases: flambe.runnable.Runnable

Base class for all runnables that are able to run on cluster.

This type of Runnables must include logic in the ‘run’ method to deal with the fact that they could be running in a distributed cluster of machines.

To provide useful information about the cluster, a RemoteEnvironment object will be injected when running remotely.

config

The secrets that the user provides. For example, ‘config[“AWS”][“ACCESS_KEY”]’

Type:configparser.ConfigParser
env

The remote environment has information about the cluster where this ClusterRunnable will be running. IMPORTANT: this object will be available only when the ClusterRunnable is running remotely.

Type:RemoteEnvironment
user_provider

The logic for specifying the user triggering this Runnable. If not passed, by default it will pick the computer’s user.

Type:Callable[[], str]
setup(self, cluster: Cluster, extensions: Dict[str, str], force: bool, **kwargs)[source]

Setup the cluster.

Parameters:
  • cluster (Cluster) – The cluster where this Runnable will be running
  • extensions (Dict[str, str]) – The ClusterRunnable extensions
  • force (bool) – The force value provided to Flambe
setup_inject_env(self, cluster: Cluster, extensions: Dict[str, str], force: bool, **kwargs)[source]

Call setup and inject the RemoteEnvironment

Parameters:
  • cluster (Cluster) – The cluster where this Runnable will be running
  • extensions (Dict[str, str]) – The ClusterRunnable extensions
  • force (bool) – The force value provided to Flambe
set_serializable_attr(self, attr, value)[source]

Set an attribute while keep supporting serializaton.