Advanced Argo file stores

Contents

Advanced Argo file stores#

Advanced Argo file stores provided by argopy are classes designed to help users to separate Input/Output operations from data analysis in their procedures.

More precisely, Argo file stores are any convenience classes providing an Argo specific and facilitated access to Argo files, with some file system logic and that are protocol agnostic. For instance, if appropriate, these classes can provide methods like open_dataset, open_json, open, ls, exists, whatever the file type and location (local folder, http, ftp or s3).

By default, these Argo file stores are instantiated using the global option gdac pointing toward the desired data source (any valid GDAC host can be used). After instantiation, a data analysis procedure can be developed, agnostic of the file source and associated access protocol.

Note that given the rather low-level implementation of these API, Argo file stores are probably for users with some advanced knowledge of the Argo dataset: analysis on raw data, Argo index files manipulation, WMO oriented procedures, etc…

These classes are all explained in the following sections:

In a nutshell#

Working on a specific float ? get simplified access to any of its netcdf dataset:

In [1]: from argopy import ArgoFloat

In [2]: ds = ArgoFloat(6903091).open_dataset('meta')

In [3]: ds
Out[3]: 
<xarray.Dataset> Size: 2MB
Dimensions:                               (N_TRANS_SYSTEM: 1,
                                           N_POSITIONING_SYSTEM: 1,
                                           N_LAUNCH_CONFIG_PARAM: 289,
                                           N_CONFIG_PARAM: 75, N_MISSIONS: 34,
                                           N_SENSOR: 12, N_PARAM: 43)
Dimensions without coordinates: N_TRANS_SYSTEM, N_POSITIONING_SYSTEM,
                                N_LAUNCH_CONFIG_PARAM, N_CONFIG_PARAM,
                                N_MISSIONS, N_SENSOR, N_PARAM
Data variables: (12/65)
    DATA_TYPE                             <U16 64B ...
    FORMAT_VERSION                        <U4 16B ...
    HANDBOOK_VERSION                      <U4 16B ...
    DATE_CREATION                         datetime64[ns] 8B ...
    DATE_UPDATE                           datetime64[ns] 8B ...
    PLATFORM_NUMBER                       int64 8B ...
    ...                                    ...
    PARAMETER_UNITS                       (N_PARAM) <U32 6kB ...
    PARAMETER_ACCURACY                    (N_PARAM) <U32 6kB ...
    PARAMETER_RESOLUTION                  (N_PARAM) <U32 6kB ...
    PREDEPLOYMENT_CALIB_EQUATION          (N_PARAM) <U4096 705kB ...
    PREDEPLOYMENT_CALIB_COEFFICIENT       (N_PARAM) <U4096 705kB ...
    PREDEPLOYMENT_CALIB_COMMENT           (N_PARAM) <U4096 705kB ...
Attributes:
    title:                Argo float metadata file
    institution:          CORIOLIS
    source:               Argo float
    history:              2025-03-10T15:36:06Z creation; 2025-04-03T22:58:05Z...
    references:           http://www.argodatamgt.org/Documentation
    user_manual_version:  3.1
    Conventions:          Argo-3.1 CF-1.6
    decoder_version:      CODA_074g
    id:                   https://doi.org/10.17882/42182

Load, curate, search most of Argo index files:

In [4]: from argopy import ArgoIndex

In [5]: idx = ArgoIndex(index_file="bgc-s")

In [6]: idx.search_wmo(6903091)
Out[6]: 
<argoindex.pandas>
Host: https://data-argo.ifremer.fr
Index: argo_synthetic-profile_index.txt.gz
Convention: argo_synthetic-profile_index (Synthetic-Profile directory file of the Argo GDAC)
In memory: True (341803 records)
Searched: True (330 matches, 0.0965%)

In [7]: df = idx.to_dataframe()

In [8]: df
Out[8]: 
                                            file  ...    profiler
0     coriolis/6903091/profiles/SR6903091_001.nc  ...  PROVOR III
1    coriolis/6903091/profiles/SR6903091_001D.nc  ...  PROVOR III
2     coriolis/6903091/profiles/SR6903091_002.nc  ...  PROVOR III
3    coriolis/6903091/profiles/SR6903091_002D.nc  ...  PROVOR III
4     coriolis/6903091/profiles/SR6903091_003.nc  ...  PROVOR III
..                                           ...  ...         ...
325  coriolis/6903091/profiles/SR6903091_163D.nc  ...  PROVOR III
326   coriolis/6903091/profiles/SR6903091_164.nc  ...  PROVOR III
327  coriolis/6903091/profiles/SR6903091_164D.nc  ...  PROVOR III
328   coriolis/6903091/profiles/SR6903091_165.nc  ...  PROVOR III
329  coriolis/6903091/profiles/SR6903091_165D.nc  ...  PROVOR III

[330 rows x 15 columns]

Provide a low-level access to a GDAC content, protocol agnostic, like a generic file system:

In [9]: from argopy import gdacfs

In [10]: fs = gdacfs("https://data-argo.ifremer.fr")

In [11]: fs.glob("dac/aoml/13857/profiles/*00*.nc")
Out[11]: 
['dac/aoml/13857/profiles/R13857_001.nc',
 'dac/aoml/13857/profiles/R13857_002.nc',
 'dac/aoml/13857/profiles/R13857_003.nc',
 'dac/aoml/13857/profiles/R13857_004.nc',
 'dac/aoml/13857/profiles/R13857_005.nc',
 'dac/aoml/13857/profiles/R13857_006.nc',
 'dac/aoml/13857/profiles/R13857_007.nc',
 'dac/aoml/13857/profiles/R13857_008.nc',
 'dac/aoml/13857/profiles/R13857_009.nc',
 'dac/aoml/13857/profiles/R13857_100.nc']
Contents