0

I have a terraform template that create aws ecs task. I filled a variable with a list of object like this:

` variables.tf

variable "microservices" {
  description = "the microservices to implement"
  type = list(object({
    name = string,
    port = number,
    secrets = optional(list(object({
      key = string,
      arn = string
    })))
  }))

`

Then in my main.tf I have the following: ` main.tf

resource "aws_ecs_task_definition" "task_definition" {
  count = length("${var.microservices}")
  family = "${var.microservices[count.index].name}-${var.environment}"
  requires_compatibilities = ["FARGATE"]
  network_mode = "awsvpc"
  cpu = 1024
  memory= 2048
  execution_role_arn = "arn:aws:iam::xxxxx:role/service-role/xxxx-test-service-role"
  container_definitions = jsonencode([
    {
      name      = "${var.microservices[count.index].name}"
      image     = "${aws_ecr_repository.microservices_ecr_repos[count.index].repository_url}"
      cpu       = 1
      essential = true
      Ulimits = [{
       Name = "nofile"
       SoftLimit = 65535
       HardLimit = 65535

      }]
      //length("${var.microservices[count.index].secrets}") > 0 ? 
      Secrets = [{ 
        Name = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].key}" : 0
        ValueFrom = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].arn}" : 0
        //Name = "${var.microservices[count.index].secrets[0].key}" 
        //ValueFrom = "${var.microservices[count.index].secrets[0].arn}" 

`

I don't understand how can I create Secrets parsing the variables. The secrets can be optional (it could exist or not). I should need a sort of for_each only in Secrets section in order to check if secret exist in input and then fill this filed.

An example of inputs is the following: `

  microservices = [
    { 
    "name" = "api", 
    "port" = 3000, 
    "secrets" = [{ "key" = "test123", "arn" = "0123"},{ "key" = "testXXX", "arn" = "1010"}] },
    { 
    "name" = "web", 
    "port" = 3000 
    "secrets" = [{ "key" = "test456", "arn" = "4567"}]
    }]

`

Anyone approach this kind of issue/configuration? What I would like to achieve is to create a task definition in aws ecs with secrets field (or empty secrets section) based on microservices input.

I tested a different data structure like here: flatten object made of nested list in terraform But in this scenario I was able to create a new data structure but when I create the resource (e.g.) aws_ecs_task_definition with a For_each it replicate some configuration like ecs tasks with the same name:

`

    locals {
   microservices_and_secrets = merge([
            for ecs_taks, group in var.microservices:
               {
                 for secrets_key, secret in group["secrets"]:                     
                       "${ecs_taks}-${secrets_key}" => {
                       name = group["name"]
                       port = group["port"]
                       secret = secret
                   }
               }
          
     ]...)
}

`

`

 

       resource "aws_ecs_task_definition" "task_definition" {
      for_each = local.microservices_and_secrets
      family = "${each.value.name}-${var.environment}" <-- ISSUE with creation because it replicates the ecs task microservice name due to foreach
      requires_compatibilities = ["FARGATE"]
      network_mode = "awsvpc"
      cpu = 1024
      memory= 2048

`

The problem is also that with this solution I can't have a microservice without any secret. e.g. the issue is the following:

`

 microservices = [
{ 
"name" = "api", 
"port" = 3000, 
"secrets" = [{ "key" = "test123", "arn" = "0123"},{ "key" = "testXXX", "arn" = "1010"}] },
{ 
"name" = "web", 
"port" = 3000 
"secrets" = [{ "key" = "test456", "arn" = "4567"}]
},
{ 
"name" = "ciaotask", 
"port" = 3000
}
]

`

`

Error: Iteration over null value
│
│   on main-aws-ecs.tf line 153, in locals:
│  152:                {
│  153:                  for secrets_key, secret in group["secrets"]:
│  154:                        "${ecs_taks}-${secrets_key}" => {
│  155:                        name = group["name"]
│  156:                        port = group["port"]
│  157:                        secret = secret
│  158:                    }
│  159:                }
│     ├────────────────
│     │ group["secrets"] is null
│
│ A null value cannot be used as the collection in a 'for' expression.

`

Anyone could help how can I manage the ecs task creation based on microservice input posted above? The question is, how can I create one aws_ecs_task_definition for each microservice present into microservices variable and it can have zero to n Secrets, starting from microservices variable list of objects.

3
  • What's wrong with the code. Any errors? Commented Nov 16, 2022 at 0:14
  • Hi @Marcin . In the code I wrote this that is wrong: code Secrets = [{ Name = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].key}" : 0 ValueFrom = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].arn}" : 0 because I put ${var.microservices[count.index].secrets[0].arn} Instead I should be able to perform (like) a for_each in secrets section checking my input variable and create this item (or do not create) based on input variable. Commented Nov 17, 2022 at 8:56
  • With the code above I get only the first item of secrets input variable array. Instead I would like to get all the items Commented Nov 17, 2022 at 9:23

1 Answer 1

0

I solved the issue. I started from this guide https://codeburst.io/how-to-securely-use-aws-secrets-manager-to-inject-secrets-into-ecs-using-infrastructure-as-code-ff2b39b420b6 then I created a template file like this:

`container_definitions.json.tpl

[{
    "name" : "${name}",
    "image": "${image}", 
    "cpu" : 1,
    "essential" : true,
    "Ulimits" : [{
        "Name" : "nofile",
        "SoftLimit" : 65535,
        "HardLimit" : 65535

    }],

    "Secrets" : ${secrets},
    "Environment" : ${environment},

    "LogConfiguration" : {
      "LogDriver" : "awslogs",
      "Options" : {
          "awslogs-group" : "${awslogs-group}",
          "awslogs-region" : "${aws_region}",
          "awslogs-stream-prefix" : "ecs"
      }
      },
    
    "portMappings" : [
      {
        "containerPort" : 3000,
        "hostPort"      : 3000
      }
    ]
  }]

`

in my main.tf instead I created the resources in this way: `

*/
data "template_file" "container_definitions" {
  count = length("${var.microservices}")
  template = file("${path.module}/template_dir/container_definitions.json.tpl")
   vars = {
     aws_region     = "${var.aws_region}"
     cpu            = 1
     image          = "${aws_ecr_repository.microservices_ecr_repos[count.index].repository_url}"
     name           = "${var.microservices[count.index].name}"
     awslogs-group  = "${aws_cloudwatch_log_group.cloudwatch_log_groups[count.index].id}"
     environment    = jsonencode("${var.microservices[count.index].environment}")
     secrets       = jsonencode("${var.microservices[count.index].secrets}")
   }
 }


/*
    AWS ECS Task definition
*/
resource "aws_ecs_task_definition" "task_definition" {
  count = length("${var.microservices}")
  family = "${var.microservices[count.index].name}-${var.environment}"
  requires_compatibilities = ["FARGATE"]
  network_mode = "awsvpc"
  cpu = "${var.microservices[count.index].cpu}"
  memory= "${var.microservices[count.index].memory}"
  execution_role_arn = "${aws_iam_role.task_execution_roles[count.index].arn}" 
  task_role_arn = "${aws_iam_role.task_execution_roles[count.index].arn}" 
  container_definitions = "${data.template_file.container_definitions[count.index].rendered}" //file("./containers_file/api.json")
}

`

In this way I was able to create a task definition in aws ecs with 0..n secrets and 0..n environment variables based on this (e.g.) input. `

  microservices = [
    { 
    "name" = "api", 
    "port" = 3000, 
    "cpu" = 1024,
    "memory" = 2048,
    "secrets" = [{ "name" = "test123", "valuefrom" = "0123"},{ "name" = "testXXX", "valuefrom" = "1010"}] },
    { 
    "name" = "web", 
    "port" = 3000,
    "cpu" = 1024,
    "memory" = 2048,
    "secrets" = [{ "name" = "test456", "valuefrom" = "4567"}],
    "environment" = [{ "name" = "weenv", "value" = "emi_is_ok" },{ "name" = "weenv123", "value" = "emi_is_ok123" } ]
    },
    { 
    "name" = "ciaotask", 
    "port" = 3000
    "cpu" = 1024,
    "memory" = 2048
    }
    ]

`

I hope this could help someone else that ran in the same issue.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.