Combining JsonPath listAt and stringAt to make a single array in AWS Steps/CDK

267 views Asked by At

I am invoking an AWS Steps orchestration - and one of the contained steps is to invoke an ECS Fargate Task.

I currently pass some of the Steps' inputs as the "command" of the ECS Task using JsonPath.listAt. I would like to also add a single extra value to the command array - but I cannot work out the syntax as a construct within CDK.

Our step's input is:

{
  "indexes": [
    "s3://bucket/test1.bam",
    "s3://bucket/test2.bam"
  ],
  "reference": "hg38"
}

And the example of us passing the indexes as command arguments to the ECS Task is:

new EcsRunTask(this, "Job", {
      integrationPattern: IntegrationPattern.RUN_JOB,
      cluster: fargateCluster,
      taskDefinition: taskDefinition,
      containerOverrides: [
        {
          command: JsonPath.listAt("$.indexes"),

I want to also pass JsonPath.stringAt("$.reference") in as the first argument of the command but I cannot get the syntax correct.

JsonPath.listAt is the only JsonPath function that returns a string[] (needed by .command in typescript CDK).

JsonPath.array can be used to build arrays - but cannot take in inputs in the same way as listAt (as in it won't flatten out an existing array input).


command: [
  JsonPath.stringAt("$.reference"),
  ...JsonPath.listAt("$.indexes"),
]

fails with

Cannot use JsonPath fields in an array, they must be used in objects


command: JsonPath.array(
            JsonPath.stringAt("$.reference"),
            ...JsonPath.listAt("$.indexes")
          ) as any

fails with

Error: Resolution error: Resolution error: Resolution error: Found an encoded list token string in a scalar string context. Use 'Fn.select(0, list)' (not 'list[0]') to extract elements from token lists..

1

There are 1 answers

2
fedonev On BEST ANSWER

The least bad option is a two step process: merge then flatten.

First, add a Pass State to merge the inputs. The state will output a nested array: [["hg38"], ["s3://bucket/test1.bam", "s3://bucket/test2.bam"]]:

const merge = new sfn.Pass(this, "merge", {
  parameters: {
    merge: sfn.JsonPath.array(
      sfn.JsonPath.array(sfn.JsonPath.stringAt("$.reference")),
      sfn.JsonPath.stringAt("$.indexes"), // CDK does not accept listAt here; the array param is of type []string
    ),
  },
  resultPath: "$.merge",
});

Now your command can use JSONPath wildcard syntax to flatten the array into the desired shape of ["hg38", "s3://bucket/test1.bam", "s3://bucket/test2.bam"]:

command: sfn.JsonPath.listAt("$.merge.merge[*][*]")

Note that this roundabout approach is needed because of how Step Functions' intrinsic functions work. It's not a limitation imposed by the CDK.