managed
Sum of the output of sizeof() for all dask keys held by the worker, both in memory and spilled to disk
managed_in_memory
Sum of the output of sizeof() for the dask keys held in RAM
managed_spilled
Sum of the output of sizeof() for the dask keys spilled to the hard drive. Note that this is the size in memory; serialized size may be different.
process
Total RSS memory measured by the OS on the worker process. This is always exactly equal to managed_in_memory + unmanaged.
unmanaged
process - managed_in_memory. This is the sum of
Python interpreter and modules
global variables
memory temporarily allocated by the dask tasks that are currently running
memory fragmentation
memory leaks
memory not yet garbage collected
memory not yet free()'d by the Python memory manager to the OS
unmanaged_old
Minimum of the 'unmanaged' measures over the last distributed.memory.recent-to-old-time
seconds
unmanaged_recent
unmanaged - unmanaged_old; in other words process memory that has been recently allocated but is not accounted for by dask; hopefully it's mostly a temporary spike.
optimistic
managed_in_memory + unmanaged_old; in other words the memory held long-term by the process under the hopeful assumption that all unmanaged_recent memory is a temporary spike
Memory readings on a worker or on the whole cluster.
Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.
Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)
SVG is more flexible but power hungry; and does not scale well to 50 + nodes.
All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them
