Reference dataset

Reference dataset#

Since Argo floats are not recovered systematically, the quality control procedure requires historical measurements to assess float data.

Historical CTD measurements have been curated and formatted by the ADMT for years, and are usually distributed as a collection of Matlab binary files. This approach has the disadvantage of requiring QC operators to download the entire reference dataset to start QC procedures, hence with the burden of having to keep the local version up to date with the last available (and most accurate and exhaustive content) version.

The argopy team has work with Coriolis to try to alleviate these limitations by uploading both reference dataset to the Ifremer erddap server. Using the erddap server, it is possible to download only the required data for a given float and the last version of the dataset.

Ship-based CTD reference measurements#

The ship-based CTD reference database is updated on a yearly bases by the Coriolis team or when there are enough new data to justify an upgrade. The ship-based CTD measurements are provided by the PIs, the ARC, Clivar Hydrographic Center and NODC/USA.

To access ship-based CTD reference measurements, we provide the CTDRefDataFetcher class.

You can use the box argument to specify a rectangular space/domain to fetch reference data for. Note that this is a list with minimum/maximum for longitude, then latitude, then pressure and finally time. The constraint on time is not mandatory.

In [1]: from argopy import CTDRefDataFetcher

In [2]: fetcher = CTDRefDataFetcher(box=[-65, -55, 10,  20, 0, 5000, '20000101', '20050101'])

In [3]: ds_ship_ctd_ref = fetcher.to_xarray()

In [4]: ds_ship_ctd_ref
Out[4]: 
<xarray.Dataset> Size: 61MB
Dimensions:          (N_POINTS: 638406)
Coordinates:
    LATITUDE         (N_POINTS) float64 5MB 15.71 15.71 15.71 ... 15.92 15.92
    LONGITUDE        (N_POINTS) float64 5MB -55.98 -55.98 ... -55.13 -55.13
    TIME             (N_POINTS) datetime64[ns] 5MB 2000-12-20T00:33:00 ... 20...
  * N_POINTS         (N_POINTS) int64 5MB 0 1 2 3 ... 638403 638404 638405
Data variables:
    PRES             (N_POINTS) float64 5MB 1.0 2.0 3.0 ... 4.999e+03 5e+03
    PSAL             (N_POINTS) float64 5MB 34.97 34.97 34.97 ... 34.85 34.85
    PTMP             (N_POINTS) float64 5MB 26.15 26.15 26.15 ... 1.487 1.487
    QCLEVEL          (N_POINTS) <U1 3MB 'O' 'O' 'O' 'O' 'O' ... 'O' 'O' 'O' 'O'
    EXPOCODE         (N_POINTS) object 5MB '24567769' '24567769' ... '51098178'
    TEMP             (N_POINTS) float64 5MB 26.15 26.15 26.15 ... 1.937 1.937
    DIRECTION        (N_POINTS) <U1 3MB 'A' 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    CYCLE_NUMBER     (N_POINTS) int64 5MB 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0
    PLATFORM_NUMBER  (N_POINTS) int64 5MB 900000 900000 900000 ... 900215 900215
Attributes:
    DATA_ID:              Reference_SHIP_CTD
    DOI:                  -
    Fetched_version:      2024V01
    Fetched_from:         erddap.ifremer.fr
    Fetched_by:           docs
    Fetched_date:         2025/04/04
    Fetched_constraints:  [x=-65.00/-55.00; y=10.00/20.00; z=0.0/5000.0; t=20...
    Fetched_uri:          https://erddap.ifremer.fr/erddap/tabledap/ArgoFloat...

Handling user name and password#

Because the ship-based CTD database may contain CTD data that are not yet freely available, access is restricted to those performing DMQC on Argo data and a user authentication is required. This can be managed by argopy.

To provide user name and a password to access this protected dataset, you must set the argopy options user and password.

These options are set automatically from environment variables ERDDAP_USERNAME and ERDDAP_PASSWORD but they can also be set for the session or within a context like this:

import argopy
argopy.set_options(user='jane_doe', password='****')

or

import argopy
from argopy import CTDRefDataFetcher

with argopy.set_options(user='jane_doe', password='****')
    ds_ship_ctd_ref = CTDRefDataFetcher(box=[-65, -55, 10,  20, 0, 5000]).to_xarray()

Warning

If you already have accessed to this dataset from the Ifremer ftp, you can use the same username and password. But if you have no idea of what to use, please email codac@ifremer.fr to request access.

Argo-based CTD reference measurements#

In areas with poor ship-based CTD data, good Argo profiles can be used for delayed mode QC. This Argo-based CTD reference database is updated by SCRIPPS (J Gilson).

To access this dataset, argopy provides the ref dataset shortname to be used with a standard DataFetcher parameterized to point toward the erddap data source:

In [5]: from argopy import DataFetcher

In [6]: fetcher = DataFetcher(ds='ref', src='erddap').region([-65, -55, 10,  20, 0, 5000])

In [7]: ds_argo_ctd_ref = fetcher.to_xarray()

In [8]: ds_argo_ctd_ref
Out[8]: 
<xarray.Dataset> Size: 5MB
Dimensions:          (N_POINTS: 65416)
Coordinates:
    LATITUDE         (N_POINTS) float64 523kB 19.91 19.91 19.91 ... 19.86 19.91
    LONGITUDE        (N_POINTS) float64 523kB -56.25 -56.25 ... -60.82 -60.82
    TIME             (N_POINTS) datetime64[ns] 523kB 2003-10-31T17:08:49 ... ...
  * N_POINTS         (N_POINTS) int64 523kB 0 1 2 3 ... 65412 65413 65414 65415
Data variables:
    CYCLE_NUMBER     (N_POINTS) int64 523kB 41 41 41 41 41 41 ... 51 52 53 54 55
    DIRECTION        (N_POINTS) <U1 262kB 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    PLATFORM_NUMBER  (N_POINTS) int64 523kB 2900199 2900199 ... 2902711 2902711
    PRES             (N_POINTS) float64 523kB 19.1 29.3 ... 2.008e+03 2.008e+03
    PSAL             (N_POINTS) float64 523kB 34.77 34.77 34.78 ... 34.61 34.61
    PTMP             (N_POINTS) float64 523kB 27.23 27.23 27.24 ... 2.282 2.266
    TEMP             (N_POINTS) float64 523kB 27.24 27.24 27.25 ... 2.426 2.409
Attributes:
    DATA_ID:              Reference_ARGO_CTD
    DOI:                  -
    Fetched_version:      ?
    Fetched_from:         erddap.ifremer.fr
    Fetched_by:           docs
    Fetched_date:         2025/04/04
    Fetched_constraints:  [x=-65.00/-55.00; y=10.00/20.00; z=0.0/500.0]
    Fetched_uri:          https://erddap.ifremer.fr/erddap/tabledap/ArgoFloat...
Contents