argopy.stores.filestore.open_mfdataset¶
- filestore.open_mfdataset(urls, concat_dim='row', max_workers: int = 112, method: str = 'thread', progress: bool = False, concat: bool = True, preprocess=None, errors: str = 'ignore', *args, **kwargs)[source]¶
Open multiple urls as a single xarray dataset.
This is a version of the
open_dataset
method that is able to handle a list of urls/paths sequentially or in parallel.Use a Threads Pool by default for parallelization.
- Parameters
- urls: list(str)
List of url/path to open
- concat_dim: str
Name of the dimension to use to concatenate all datasets (passed to
xarray.concat
)- max_workers: int
Maximum number of threads or processes
- method: str
- The parallelization method to execute calls asynchronously:
thread
(Default): use a pool of at mostmax_workers
threadsprocess
: use a pool of at mostmax_workers
processes(XFAIL) a
distributed.client.Client
object (distributed.client.Client
)
Use ‘seq’ to simply open data sequentially
- progress: bool
Display a progress bar (True by default)
- preprocess: callable (optional)
If provided, call this function on each dataset prior to concatenation
- errors: str
Should it ‘raise’ or ‘ignore’ errors. Default: ‘ignore’
- Returns