argopy.stores.httpstore.open_mfdataset#

httpstore.open_mfdataset(urls, max_workers: int = 6, method: str = 'thread', progress: bool | str = False, concat: bool = True, concat_dim='row', preprocess=None, preprocess_opts={}, open_dataset_opts={}, errors: str = 'ignore', compute_details: bool = False, *args, **kwargs)[source]#

Open multiple urls as a single xarray dataset.

This is a version of the argopy.stores.httpstore.open_dataset method that is able to handle a list of urls/paths sequentially or in parallel.

Use a Threads Pool by default for parallelization.

Parameters:

  • urls (list(str)) – List of url/path to open

  • max_workers (int, default: 6) – Maximum number of threads or processes

  • method (str, default: thread) –

    The parallelization method to execute calls asynchronously:

    • thread (default): use a pool of at most max_workers threads

    • process: use a pool of at most max_workers processes

    • distributed.client.Client: Experimental, expect this method to fail !

    • seq: open data sequentially, no parallelization applied

    • erddap: use a pool of at most max_workers threads, comes with a nice dashboard dedicated

      to erddap server requests.

  • progress (bool, default: False) – Display a progress bar

  • concat (bool, default: True) – Concatenate results in a single xarray.Dataset or not (in this case, function will return a list of xarray.Dataset)

  • concat_dim (str, default: row) – Name of the dimension to use to concatenate all datasets (passed to xarray.concat)

  • preprocess (callable (optional)) – If provided, call this function on each dataset prior to concatenation

  • preprocess_opts (dict (optional)) – If preprocess is provided, pass this as options

  • errors (str, default: ignore) –

    Define how to handle errors raised during data URIs fetching:

    • raise: Raise any error encountered

    • ignore (default): Do not stop processing, simply issue a debug message in logging console

    • silent: Do not stop processing and do not issue log message

  • kwargs (Other args and) –

Returns:

output

Return type:

xarray.Dataset or list of xarray.Dataset