Edit this page on GitHub

Home > docs > plugins > S3 Task

S3 Task

The s3 task allows users to work with S3-compatible object stores.

Usage

To be able to use the task in a Concord flow, it must be added as a dependency:

configuration:
  dependencies:
  - mvn://com.walmartlabs.concord.plugins:s3-task:1.20.1

This adds the task to the classpath and allows you to invoke the task in a flow:

Common Parameters

  • action - (mandatory) action to perform:
    • getObject - download an object from the store;
    • putObject - upload an object to the store.
  • endpoint - string, the store API endpoint, if not specified the default Amazon S3 endpoint is used;
  • region - (mandatory) string, the store’s region;
  • pathStyleAccess - boolean, enables the path-style access which is used by some of S3 implementations;
  • auth - (mandatory) authentication parameters:
    • basic
      • accessKey - string, the endpoint’s access key;
      • secretKey - string, the endpoint’s secret key;
  • bucketName - (mandatory) string, name of the object’s bucket;
  • objectKey - (mandatory) string, the object’s key;
  • ignoreErrors - boolean, if true any errors that occur during the execution will be ignored and stored in the result variable;

Get Object

The getObject action is used to download objects from the store and save them as temporary files in the process’ working directory.

- task: s3
  in:
    action: "getObject"
    region: "us-east-1"
    auth:
      basic:
        accessKey: "..."
        secretKey: "..."
    bucketName: "myBucket"
    objectKey: "myObject"

- log: "Saved as ${result.path}"

The result is stored in the result variable:

  • ok - boolean, true if the execution is successful;
  • path - string, path of the downloaded object;
  • error - string, error message (if ignoreErrors is false).

Put Object

The putObject action is used to upload files from the process’ working directory into the object store.

- task: s3
  in:
    action: "putObject"
    region: "us-east-1"
    auth:
      basic:
        accessKey: "..."
        secretKey: "..."
    bucketName: "myBucket"
    objectKey: "myObject"
    src: "local.file"

Examples