Base class for simple web clients for the data retriever module. This class is mainly designed for downloading data from FTP and HTTP type data sources, although should in theory be general enough to get data from any web service.

This class has two user facing methods search and fetch the former generates a set of results for files available through the service the client is querying and the latter downloads that data.

The search method takes a set of objects and then converts these into a call to _get_url_for_timerange. It does this through the map_ dictionary which represents the attrs objects as a dictionary.

Methods Summary

fetch(qres[, path, overwrite, progress, …])

Download a set of results.

search(*args, **kwargs)

Query this client for a list of results.

Methods Documentation

fetch(qres, path=None, overwrite=False, progress=True, downloader=None, wait=True)[source]

Download a set of results.

  • qres (QueryResponse) – Results to download.

  • path (str or pathlib.Path, optional) – Path to the download directory, or file template including the {file} string which will be replaced with the filename.

  • overwrite (bool or str, optional) – Determine how to handle downloading if a file already exists with the same name. If False the file download will be skipped and the path returned to the existing file, if True the file will be downloaded and the existing file will be overwritten, if 'unique' the filename will be modified to be unique.

  • progress (bool, optional) – If True show a progress bar showing how many of the total files have been downloaded. If False, no progress bar will be shown.

  • downloader (parfive.Downloader, optional) – The download manager to use.

  • wait (bool, optional) – If False will not be called. Only has any effect if downloader is not None.


results (parfive.Results)

search(*args, **kwargs)[source]

Query this client for a list of results.


*args (tuple) – objects representing the query.