argopy.stores.httpstore.open_mfjson

httpstore.open_mfjson(urls, max_workers=112, method: str = 'thread', progress: bool = False, preprocess=None, errors: str = 'ignore', *args, **kwargs)

Open multiple json urls

This is a parallelized version of open_json. Use a Threads Pool by default for parallelization.

Parameters

urls: list(str)

max_workers: int

Maximum number of threads or processes.

method:

The parallelization method to execute calls asynchronously:

• ‘thread’ (Default): use a pool of at most max_workers threads

• ‘process’: use a pool of at most max_workers processes

• (XFAIL) Dask client object: use a Dask distributed client object

Use ‘seq’ to simply open data sequentially

progress: bool

Display a progress bar (True by default, not for dask client method)

preprocess: (callable, optional)

If provided, call this function on each json set

Returns

list()