argopy.stores.httpstore.open_mfjson¶
-
httpstore.
open_mfjson
(urls, max_workers=112, method: str = 'thread', progress: bool = False, preprocess=None, errors: str = 'ignore', *args, **kwargs)[source]¶ Open multiple json urls
This is a parallelized version of
open_json
. Use a Threads Pool by default for parallelization.- Parameters
- urls: list(str)
- max_workers: int
Maximum number of threads or processes.
- method:
- The parallelization method to execute calls asynchronously:
‘thread’ (Default): use a pool of at most
max_workers
threads‘process’: use a pool of at most
max_workers
processes(XFAIL) Dask client object: use a Dask distributed client object
Use ‘seq’ to simply open data sequentially
- progress: bool
Display a progress bar (True by default, not for dask client method)
- preprocess: (callable, optional)
If provided, call this function on each json set
- Returns
- list()