A worker plugin enables custom code to run at different stages of the Workers' lifecycle: at setup, during task state transitions, when a task or dependency is released, and at teardown.

A plugin enables custom code to run at each of step of a Workers's life. Whenever such an event happens, the corresponding method on this class will be called. Note that the user code always runs within the Worker's main thread.

To implement a plugin implement some of the methods of this class and register the plugin to your client in order to have it attached to every existing and future workers with Client.register_worker_plugin .

Interface to extend the Worker

Examples

This example is valid syntax, but we were not able to check execution

>>> class ErrorLogger(WorkerPlugin):
...     def __init__(self, logger):
...         self.logger = logger
...
...     def setup(self, worker):
...         self.worker = worker
...
...     def transition(self, key, start, finish, *args, **kwargs):
...         if finish == 'error':
...             ts = self.worker.tasks[key]
...             exc_info = (type(ts.exception), ts.exception, ts.traceback)
...             self.logger.error(
...                 "Error during computation of '%s'.", key,
...                 exc_info=exc_info
...             )

This example is valid syntax, but we were not able to check execution

>>> plugin = ErrorLogger()
... client.register_worker_plugin(plugin)  # doctest: +SKIP

See :

Back References

The following pages refer to to this document either explicitly or contain code examples using this.

distributed.diagnostics.plugin.WorkerPlugin distributed.diagnostics.plugin.NannyPlugin distributed.client.Client.register_worker_plugin

Local connectivity graph

Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.

Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)

SVG is more flexible but power hungry; and does not scale well to 50 + nodes.

All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them

---

File: /distributed/diagnostics/plugin.py#97
type: <class 'type'>
Commit: