argopy.stores.httpstore.open_mfjson

argopy.stores.httpstore.open_mfjson

httpstore.open_mfjson(urls, max_workers=112, method: str = 'thread', progress: bool = False, preprocess=None, errors: str = 'ignore', *args, **kwargs)[source]

Open multiple json urls

This is a parallelized version of open_json. Use a Threads Pool by default for parallelization.

Parameters

  • urls (list(str)) –

  • max_workers (int) – Maximum number of threads or processes.

  • method –

    The parallelization method to execute calls asynchronously:

    • ’thread’ (Default): use a pool of at most max_workers threads

    • ’process’: use a pool of at most max_workers processes

    • (XFAIL) Dask client object: use a Dask distributed client object

    Use ‘seq’ to simply open data sequentially

  • progress (bool) – Display a progress bar (True by default, not for dask client method)

  • preprocess ((callable, optional)) – If provided, call this function on each json set

Return type

list()