argopy.stores.index.implementations.index_s3.s3index

class s3index

A prototype for an Argo index store relying on remote CSV or PQ index data on S3.

The index is scanned/searched directly on the s3 server using the boto3.client.select_object_content boto3 method.

The key limitation here is that we can only search for WMO and CYC. This is due to the fact that G. MAZE could not manage to convert (CAST) latitude, longitude and time to a more appropriate data type to execute a search. All variables are considered string by default.

Examples

idx = s3index()

idx.search_wmo(6903091) idx.search_wmo(6903091) idx.search_wmo([13857, 6903091]) idx.search_cyc(1) idx.search_cyc(1, nrows=100) idx.search_cyc([0, 1]) idx.search_cyc([0, 1], nrows=100) idx.search_wmo_cyc(6903091, 1) idx.search_wmo_cyc(6903091, [1, 2]) idx.search_wmo_cyc([13857, 6903091], 1) idx.search_wmo_cyc([13857, 6903091], [1, 2])

idx.pd # Return search results as a pd.DataFrame idx.pq # Return search results as a pa.Table

idx.stats # Data processing stats

__init__()

Methods

__init__()
query(sql_expression)
run()
search_cyc(CYCs[, nrows])
search_wmo(WMOs[, nrows])
search_wmo_cyc(WMOs, CYCs[, nrows])

Attributes

bucket_name	Name of the S3 bucket
pd	Return search result as a pd.DataFrame
pq	Return search result as a pa.Table
sql_formaters	SQL syntax formatters for all search methods and for each strategy