API reference

All the objects below are importable from copernicusmarine_delivery.

Operations

pydantic model copernicusmarine_delivery.Upload

Upload files to the given dataset and product.

Parameters:
  • files – Relative or absolute path to local files. See Files paths and anchors in the documentation.

  • max_concurrent_uploads – The maximum number of parallel threads that will be used to upload files defined in the files attribute. Defaults to 5.

  • chunk_size_mb – The chunk size (in MB) in which the files will be split into for multipart uploads. Defaults to 16 MB.

  • chunk_concurrency – The number of chunks per file that will be uploaded in parallel in multipart uploads. Defaults to 6.

  • dry_run – Validate the upload without performing any actual operation in S3.

add(file: str) → None

Add a file to the operation.

Parameters:

file – Path of the file. See File paths and anchors in the documentation.

submit(product_id: str, dataset_id: str, anchor: str | None = None, raise_on_upload_error: bool = False, max_concurrent_uploads: int = 5, chunk_size_mb: int = 16, chunk_concurrency: int = 6, dry_run: bool = False, disable_progress_bar: bool = False, staging: bool = False, **kwargs) → ResponseUpload

Perform the upload.

Parameters:
  • product_id – The ID of the product.

  • dataset_id – The ID of the dataset.

  • anchor – Optional anchor to convert the local file path to an S3 suffix. See File paths and anchors in the documentation.

  • raise_on_upload_error – If True, raise an exception and stop the upload if any file fails to upload. By default, the upload will continue and skip any files that fail to upload.

  • max_concurrent_uploads – The maximum number of parallel threads that will be used to upload files defined in the operations attribute. Defaults to 5.

  • chunk_size_mb – The chunk size (in MB) in which the files will be split into for multipart uploads. Defaults to 16 MB.

  • chunk_concurrency – The number of chunks per file that will be uploaded in parallel in multipart uploads. Defaults to 6.

  • staging – Use the staging environment instead of production.

Returns:

ResponseUpload object containing the result of the upload.

field anchor: str | None = None
field files: list[str] [Required]
field operation: Literal[OperationNames.upload] = OperationNames.upload
pydantic model copernicusmarine_delivery.Delete

Delete files from the given dataset and product.

Parameters:
  • files – Files’ Key suffix in S3 without product_id/dataset_id in the folder structure. See File paths and anchors in the documentation for more details.

  • dry_run – Validate the delete without performing any actual operation in S3.

add(file: str) → None

Add a file to the operation.

Parameters:

file – Path of the file. See File paths and anchors in the documentation.

submit(product_id: str, dataset_id: str, dry_run: bool = False, staging: bool = False, **kwargs) → ResponseDelete

Perform the delete.

Parameters:
  • product_id – The ID of the product.

  • dataset_id – The ID of the dataset.

  • staging – Use the staging environment instead of production.

Returns:

ResponseDelete object containing the result of the delete.

field files: list[str] [Required]
field operation: Literal[OperationNames.delete] = OperationNames.delete
pydantic model copernicusmarine_delivery.Delivery

Creates a delivery. A delivery is a set of operations (upload or delete) to be performed on a dataset.

Each operation will be performed sequentially in the order they are given. Each operation can have multiple sources (files) to be processed.

Parameters:
  • operations – A list of operations to be performed. Each operation is a tuple with the operation name as the first element and a list of sources as the second element. Available operations are ‘upload’ and ‘delete’.

  • max_concurrent_uploads – The maximum number of parallel threads that will be used to upload files defined in the operations attribute. Defaults to 5.

  • chunk_size_mb – The chunk size (in MB) in which the files will be split into for multipart uploads. Defaults to 16 MB.

  • chunk_concurrency – The number of chunks per file that will be uploaded in parallel in multipart uploads. Defaults to 6.

  • dry_run – Validate the delivery without performing any actual operation in S3.

add(operation: Upload | Delete) → None

Add an operation to the delivery. Operations are performed in the order they are added.

Parameters:

operation – An Upload or Delete operation.

submit(product_id: str, dataset_id: str, anchor: str | None = None, raise_on_upload_error: bool = False, max_concurrent_uploads: int = 5, chunk_size_mb: int = 16, chunk_concurrency: int = 6, dry_run: bool = False, disable_progress_bar: bool = False, staging: bool = False, **kwargs) → ResponseDelivery

Perform the delivery by executing all added operations sequentially.

Parameters:
  • product_id – The ID of the product.

  • dataset_id – The ID of the dataset.

  • anchor – Optional anchor to convert the local file path to an S3 suffix. See File paths and anchors in the documentation for more details.

  • raise_on_upload_error – If True, raise an exception and stop the delivery if any file fails to upload. By default, the delivery will continue and skip any files that fail to upload.

  • max_concurrent_uploads – The maximum number of parallel threads that will be used to upload files defined in the operations attribute. Defaults to 5.

  • chunk_size_mb – The chunk size (in MB) in which the files will be split into for multipart uploads. Defaults to 16 MB.

  • chunk_concurrency – The number of chunks per file that will be uploaded in parallel in multipart uploads. Defaults to 6.

  • staging – Use the staging environment instead of production.

Returns:

ResponseDelivery object containing the result of the delivery.

field operations: list[Upload | Delete] [Optional]

Delivery status

copernicusmarine_delivery.delivery_status(delivery_id: str, on_behalf_of: str | None = None, staging: bool = False) → Delivery

Get the status of a delivery.

Parameters:
  • delivery_id – The ID of the delivery to check.

  • on_behalf_of – For internal usage only.

  • staging – Use the staging environment instead of production.

Returns:

DeliveryModel object representing the status of the delivery.

copernicusmarine_delivery.list_deliveries(timeframe: Literal['s', 'm', 'h', 'd', 'w', 'mo', 'y', 'all'] = 'mo', on_behalf_of: str | None = None, staging: bool = False) → list[Delivery]

List all deliveries for a given pushing entity.

Parameters:
  • timeframe – The timeframe to filter deliveries by. Defaults to “mo” (last month).

  • staging – Use the staging environment instead of production.

Returns:

A list of DeliveryModel objects sorted by delivery_id in descending order (most recent first).

Models

pydantic model copernicusmarine_delivery.ResponseUpload

Metadata returned when using upload()

field dataset_id: str | None = None

Dataset ID.

field delivery_id: str | None = None

Delivery ID.

field dry_run: bool | None = None

Dry run flag, if True, no actual operation was performed in S3.

field fatal_error: str | None = None

Any error that may prematurely stop the delivery.

field files_failed: list[ErrorFile] [Optional]

Potential I/O errors, might be on the user side, not necessarily user fault

field files_uploaded: list[tuple[Path, S3KeySuffix]] [Optional]

Successful uploaded file names

field product_id: str | None = None

Product ID.

field pushing_entity_id: str | None = None

Pushing entity ID.

pydantic model copernicusmarine_delivery.ResponseDelete

Metadata returned when using delete()

field dataset_id: str | None = None

Dataset ID.

field delivery_id: str | None = None

Delivery ID.

field dry_run: bool | None = None

Dry run flag, if True, no actual operation was performed in S3.

field fatal_error: str | None = None

Any error that may prematurely stop the delivery.

field files_to_delete: list[DeleteFile] [Optional]
field product_id: str | None = None

Product ID.

field pushing_entity_id: str | None = None

Pushing entity ID.

pydantic model copernicusmarine_delivery.ResponseDelivery

Metadata returned when using delivery()

field dataset_id: str | None = None

Dataset ID.

field delivery_id: str | None = None

Delivery ID.

field dry_run: bool | None = None

Dry run flag, if True, no actual operation was performed in S3.

field fatal_error: str | None = None

Any error that may prematurely stop the delivery.

field operations_responses: list[ResponseUpload | ResponseDelete] [Optional]

List of responses for each operation in the delivery.

field product_id: str | None = None

Product ID.

field pushing_entity_id: str | None = None

Pushing entity ID.

copernicusmarine_delivery.DeliveryModel

alias of Delivery

class copernicusmarine_delivery.DeliveryStatus(*values)
failed = 'failed'
partially_failed = 'partially-failed'
pending = 'pending'
published = 'published'
validated = 'validated'

Exceptions

exception copernicusmarine_delivery.AuthenticationError

Authentication Error is raised if the token response from the Keycloak service returns >= 400

exception copernicusmarine_delivery.InvalidDeliveryIdsError(message: str)

Raised when one or more IDs in the delivery are not expected.

exception copernicusmarine_delivery.InvalidDeliverySizeError(limit_raised: str, details: str)

Raised when either the number of files, or a file size or the total delivery size exceed the limits.

exception copernicusmarine_delivery.InvalidFilesError(invalid_files: list[InvalidFile])

Raised when an error occurs during the validation of files. Will list the files and their errors that failed validation.

exception copernicusmarine_delivery.NoSuccessfulUploadsError(error_files: list[ErrorFile])

Raised when an error occurs during the upload process. Will list the files and their errors that failed to upload.