Processes v1

• Directory Structure
• Additional Concord Files
• DSL
• Public Flows
• Variables
  • Provided Variables
  • Context
  • Output Variables

Directory Structure

Regardless of how the process was started – using a project and a Git repository or by sending a payload archive, Concord assumes a certain structure of the process’s working directory:

• concord.yml: a Concord DSL file containing the main flow, configuration, profiles and other declarations;
• concord/*.yml: directory containing extra Concord YAML files;
• forms: directory with custom forms.

Anything else is copied as-is and available for the process. Plugins can require other files to be present in the working directory.

The same structure should be used when storing your project in a Git repository. Concord clones the repository and recursively copies the specified directory path (/ by default which includes all files in the repository) to the working directory for the process. If a subdirectory is specified in the Concord repository’s configuration, any paths outside the configuration-specified path are ignored and not copied. The repository name it not included in the final path.

Additional Concord Files

The default use case with the Concord DSL is to maintain everything in the one concord.yml file. The usage of a concord folder and files within it allows you to reduce the individual file sizes.

./concord/test.yml:

configuration:
  arguments:
    nested:
      name: "stranger"
flows:
  default:
  - log: "Hello, ${nested.name}!"

./concord.yml:

configuration:
  arguments:
    nested:
      name: "Concord"

The above example prints out Hello, Concord!, when running the default flow.

Concord folder merge rules:

• Files are loaded in alphabetical order, including subdirectories.
• Flows and forms with the same names are overridden by their counterpart from the files loaded previously.
• All triggers from all files are added together. If there are multiple trigger definitions across several files, the resulting project contains all of them.
• Configuration values are merged. The values from the last loaded file override the values from the files loaded earlier.
• Profiles with flows, forms and configuration values are merged according to the rules above.

DSL

Concord DSL files contain configuration, flows, profiles and other declarations.

The top-level syntax of a Concord DSL file is:

configuration:
  ...

flows:
  ...

publicFlows:
  ...

forms:
  ...

triggers:
  ...

profiles:
  ...

resources:
  ...

imports:
  ...

Let’s take a look at each section:

• configuration - defines process configuration, dependencies, arguments and other values;
• flows - contains one or more Concord flows;
• publicFlows - list of flow names which may be used as an entry point;
• forms - Concord form definitions;
• triggers - contains trigger definitions;
• profiles - declares profiles that can override declarations from other sections;
• resources - configurable paths to Concord resources;
• imports - allows referencing external Concord definitions.

Public Flows

Flows listed in the publicFlows section are the only flows allowed as entry point values. This also limits the flows listed in the repository run dialog. When the publicFlows is omitted, all flows are considered public.

Flows from an imported repository are subject to the same setting. publicFlows defined in the imported repository are merged with those defined in the main repository.

publicFlows:
  - default
  - enterHere

flows:
  default:
    - log: "Hello!"
    - call: internalFlow

  enterHere:
    - "Using alternative entry point."

  # not listed in the UI repository start popup
  internalFlow:
    - log: "Only callable from another flow."

Variables

Process arguments, saved process state and automatically provided variables are exposed as flow variables:

flows:
  default:
    - log: "Hello, ${initiator.displayName}"

In the example above the expression ${initator.displayName} references an automatically provided variable inititator and retrieves it’s displayName field value.

Flow variables can be defined using the DSL’s set step, the arguments section in the process configuration, passed in the API request when the process is created, etc.

Provided Variables

Concord automatically provides several built-in variables upon process execution in addition to the defined variables:

• execution or context: a reference to the current execution’s context, instance of com.walmartlabs.concord.sdk.Context;
• txId - an unique identifier of the current process;
• parentInstanceId - an identifier of the parent process;
• tasks - allows access to available tasks (for example: ${tasks.get('oneops')});
• workDir - path to the working directory of a current process;
• initiator - information about the user who started a process:
  • initiator.username - login, string;
  • initiator.displayName - printable name, string;
  • initiator.email - email address, string;
  • initiator.groups - list of user’s groups;
  • initiator.attributes - other LDAP attributes; for example initiator.attributes.mail contains the email address.
• currentUser - information about the current user. Has the same structure as initiator;
• requestInfo - additional request data (see the note below):
  • requestInfo.query - query parameters of a request made using user-facing endpoints (e.g. the portal API);
  • requestInfo.ip - client IP address, where from request is generated.
  • requestInfo.headers - headers of request made using user-facing endpoints.
• projectInfo - project’s data:
  • projectInfo.orgId - the ID of the project’s organization;
  • projectInfo.orgName - the name of the project’s organization;
  • projectInfo.projectId - the project’s ID;
  • projectInfo.projectName - the project’s name;
  • projectInfo.repoId - the project’s repository ID;
  • projectInfo.repoName - the repository’s name;
  • projectInfo.repoUrl - the repository’s URL;
  • projectInfo.repoBranch - the repository’s branch;
  • projectInfo.repoPath - the repository’s path (if configured);
  • projectInfo.repoCommitId - the repository’s last commit ID;
  • projectInfo.repoCommitAuthor - the repository’s last commit author;
  • projectInfo.repoCommitMessage - the repository’s last commit message.
• processInfo - the current process’ data:
  • processInfo.activeProfiles - list of active profiles used for the current execution;
  • processInfo.sessionToken - the current process’ session token can be used to call Concord API from flows.

LDAP attributes must be allowed in the configuration.

Note: only the processes started using the browser link provide the requestInfo variable. In other cases (e.g. processes triggered by GitHub) the variable might be undefined or empty.

Availability of other variables and “beans” depends on the installed Concord plugins and the arguments passed in at the process invocation and stored in the request data.

Context

The context variable provides access to the current process’ state: variables, current flow name, etc. The context variable is available at any moment during the flow execution and can be accessed using expressions, scripts or in tasks:

flows:
  default:
    - log: "All variables: ${context.toMap()}"

    - script: javascript
      body: |
        var allVars = execution.toMap();
        print('Getting all variables in a JavaScript snippet: ' + allVars);

Note: in the script environment the context variable called execution to avoid clashes with the JSR 223 scripting context.

Output Variables

Concord has the ability to return process data when a process completes. The names of returned variables should be declared in the configuration section:

configuration:
  out:
    - myVar1

Output variables may also be declared dynamically using multipart/form-data parameters if allowed in a Project’s configuration. CAUTION: this is a not secure if secret values are stored in process variables

$ curl ... -F out=myVar1 https://concord.example.com/api/v1/process
{
  "instanceId" : "5883b65c-7dc2-4d07-8b47-04ee059cc00b"
}

Retrieve the output variable value(s) after the process finishes:

# wait for completion...
$ curl .. https://concord.example.com/api/v2/process/5883b65c-7dc2-4d07-8b47-04ee059cc00b
{
  "instanceId" : "5883b65c-7dc2-4d07-8b47-04ee059cc00b",
  "meta": {
    out" : {
      "myVar1" : "my value"
    },
  }  
}

It is also possible to retrieve a nested value:

configuration:
  out:
    - a.b.c

flows:
  default:
    - set:
        a:
          b:
            c: "my value"
            d: "ignored"

$ curl ... -F out=a.b.c https://concord.example.com/api/v1/process

In this example, Concord looks for variable a, its field b and the nested field c.

Additionally, the output variables can be retrieved as a JSON file:

$ curl ... https://concord.example.com/api/v1/process/5883b65c-7dc2-4d07-8b47-04ee059cc00b/attachment/out.json

{"myVar1":"my value"}

Any value type that can be represented as JSON is supported.