register_worker_plugin(self, plugin=None, name=None, nanny=None, **kwargs)
This registers a new object to handle setup, task state transitions and teardown for workers in this cluster. The plugin will instantiate itself on all currently connected workers. It will also be run on any worker that connects in the future.
The plugin may include methods setup
, teardown
, transition
, and release_key
. See the dask.distributed.WorkerPlugin
class or the examples below for the interface and docstrings. It must be serializable with the pickle or cloudpickle modules.
If the plugin has a name
attribute, or if the name=
keyword is used then that will control idempotency. If a plugin with that name has already been registered then any future plugins will not run.
For alternatives to plugins, you may also wish to look into preload scripts.
The plugin object to register.
A name for the plugin. Registering a plugin with the same name will have no effect. If plugin has no name attribute a random name is used.
Whether to register the plugin with workers or nannies.
If you pass a class as the plugin, instead of a class instance, then the class will be instantiated with any extra keyword arguments.
Registers a lifecycle worker plugin for all current and future workers.
>>> class MyPlugin(WorkerPlugin):This example is valid syntax, but we were not able to check execution
... def __init__(self, *args, **kwargs):
... pass # the constructor is up to you
... def setup(self, worker: dask.distributed.Worker):
... pass
... def teardown(self, worker: dask.distributed.Worker):
... pass
... def transition(self, key: str, start: str, finish: str, **kwargs):
... pass
... def release_key(self, key: str, state: str, cause: str | None, reason: None, report: bool):
... pass
>>> plugin = MyPlugin(1, 2, 3)
... client.register_worker_plugin(plugin)
You can get access to the plugin with the get_worker
function
>>> client.register_worker_plugin(other_plugin, name='my-plugin')This example is valid syntax, but we were not able to check execution
... def f():
... worker = get_worker()
... plugin = worker.plugins['my-plugin']
... return plugin.my_state
>>> future = client.run(f)See :
The following pages refer to to this document either explicitly or contain code examples using this.
distributed.client.Client.unregister_worker_plugin
Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.
Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)
SVG is more flexible but power hungry; and does not scale well to 50 + nodes.
All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them