argopy.stores.s3store.open_mfdataset

s3store.open_mfdataset(urls, max_workers: int = 6, method: str = 'thread', progress: bool | str = False, concat: bool = True, concat_dim='row', preprocess=None, preprocess_opts={}, open_dataset_opts={}, errors: str = 'ignore', compute_details: bool = False, *args, **kwargs)

Open multiple urls as a single xarray dataset.

This is a version of the argopy.stores.httpstore.open_dataset method that is able to handle a list of urls/paths sequentially or in parallel.

Use a Threads Pool by default for parallelization.

Parameters:

• urls (list(str)) – List of url/path to open

• max_workers (int, default: 6) – Maximum number of threads or processes

• method (str, default: thread) –

  The parallelization method to execute calls asynchronously:

  • thread (default): use a pool of at most max_workers threads

  • process: use a pool of at most max_workers processes

  • distributed.client.Client: Experimental, expect this method to fail !

  • seq: open data sequentially, no parallelization applied

  • erddap: use a pool of at most max_workers threads, comes with a nice dashboard dedicated

    to erddap server requests.

• progress (bool, default: False) – Display a progress bar

• concat (bool, default: True) – Concatenate results in a single xarray.Dataset or not (in this case, function will return a list of xarray.Dataset)

• concat_dim (str, default: row) – Name of the dimension to use to concatenate all datasets (passed to xarray.concat)

• preprocess (callable (optional)) – If provided, call this function on each dataset prior to concatenation

• preprocess_opts (dict (optional)) – If preprocess is provided, pass this as options

• errors (str, default: ignore) –

  Define how to handle errors raised during data URIs fetching:

  • raise: Raise any error encountered

  • ignore (default): Do not stop processing, simply issue a debug message in logging console

  • silent: Do not stop processing and do not issue log message

• kwargs (Other args and)

Returns:

output

Return type:

xarray.Dataset or list of xarray.Dataset