run(self) -> 'Generator[tuple[str, TaskState, set[WorkerState] | None], WorkerState | None, None]'
"replicate", <TaskState>, None
"replicate", <TaskState>, {subset of potential workers to replicate to}
"drop", <TaskState>, None
"drop", <TaskState>, {subset of potential workers to drop from}
Each element yielded indicates the desire to create or destroy a single replica of a key. If a subset of workers is not provided, it defaults to all workers on the cluster. Either the ActiveMemoryManager or the Worker may later decide to disregard the request, e.g. because it would delete the last copy of a key or because the key is currently needed on that worker.
You may optionally retrieve which worker it was decided the key will be replicated to or dropped from, as follows:
.. code-block:: python
choice = (yield "replicate", ts, None)
choice
is either a WorkerState or None; the latter is returned if the ActiveMemoryManager chose to disregard the request.
The current pending (accepted) commands can be inspected on self.manager.pending
; this includes the commands previously yielded by this same method.
The current memory usage on each worker, downstream of all pending commands, can be inspected on self.manager.workers_memory
.
This method is invoked by the ActiveMemoryManager every few seconds, or whenever the user invokes scheduler.amm_run_once(). It is an iterator that must emit any of the following:
Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.
Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)
SVG is more flexible but power hungry; and does not scale well to 50 + nodes.
All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them