astropy 5.0

NotesParametersReturnsBackRef
download_files_in_parallel(urls, cache='update', show_progress=True, timeout=None, sources=None, multiprocessing_start_method=None, pkgname='astropy')

Blocks until all files have downloaded. The result is a list of local file paths corresponding to the given urls.

The results will be stored in the cache under the values in urls even if they are obtained from some other location via sources . See :None:None:`~download_file` for details.

Notes

If a URL is unreachable, the downloading will grind to a halt and the exception will propagate upward, but an unpredictable number of files will have been successfully downloaded and will remain in the cache.

Parameters

urls : list of str

The URLs to retrieve.

cache : bool or "update", optional

Whether to use the cache (default is :None:None:`True`). If "update", always download the remote URLs to see if new data is available and store the result in cache.

versionchanged

The default was changed to "update" and setting it to False will print a Warning and set it to "update" again, because the function will not work properly without cache. Using True will work as expected.

versionchanged

The default was changed to True and setting it to False will print a Warning and set it to True again, because the function will not work properly without cache.

show_progress : bool, optional

Whether to display a progress bar during the download (default is :None:None:`True`)

timeout : float, optional

Timeout for each individual requests in seconds (default is the configurable :None:None:`astropy.utils.data.Conf.remote_timeout`).

sources : dict, optional

If provided, for each URL a list of URLs to try to obtain the file from. The result will be stored under the original URL. For any URL in this dictionary, the original URL will not be tried unless it is in this list; this is to prevent long waits for a primary server that is known to be inaccessible at the moment.

multiprocessing_start_method : str, optional

Useful primarily for testing; if in doubt leave it as the default. When using multiprocessing, certain anomalies occur when starting processes with the "spawn" method (the only option on Windows); other anomalies occur with the "fork" method (the default on Linux).

pkgname : `str`, optional

The package name to use to locate the download cache. i.e. for pkgname='astropy' the default cache location is ~/.astropy/cache .

Returns

paths : list of str

The local file paths corresponding to the downloaded URLs.

Download multiple files in parallel from the given URLs.

Examples

See :

Back References

The following pages refer to to this document either explicitly or contain code examples using this.

astropy.io.fits.hdu.hdulist.fitsopen astropy.utils.data.export_download_cache

Local connectivity graph

Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.

Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)

SVG is more flexible but power hungry; and does not scale well to 50 + nodes.

All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them


File: /astropy/utils/data.py#1474
type: <class 'function'>
Commit: