This is the base class for indexable Blockwise-IO arguments. When constructing a Blockwise
Layer, one or more of the collection tuples passed in with indices
may contain a BlockwiseDep
instance (in place of a "real" collection name). This allows a new collection to be created (via IO) within a Blockwise
layer.
All BlockwiseDep
instances must define a numblocks
attribute to speficy the number of blocks/partitions the object can support along each dimension. The object should also define a produces_tasks
attribute to specify if any nested tasks will be passed to the Blockwise function.
Blockwise-IO argument
The following pages refer to to this document either explicitly or contain code examples using this.
dask.blockwise.BlockwiseDepDict
Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.
Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)
SVG is more flexible but power hungry; and does not scale well to 50 + nodes.
All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them