The provided Terraform code defines Helm charts and infrastructure components for managing EKS cluster along with associated resources such as IAM roles, Kubernetes namespaces, and monitoring according to Automat-IT best practices. It provides infrastructure components (like Ingress, load balancing, scaling, monitoring, secrets, and DNS) necessary for setting up and managing EKS cluster.

Note: Main branch is following Terraform AWS 6.x Provider Version, and latest version of EKS. To work with 5.x terraform provider version please checkout on - main-terraform-aws-v5-provider

Include a reference to the directory of your Terraform environment where you configured the Amazon Elastic Kubernetes Service (EKS) cluster setup and set correct variables.

Reference values could be found at examples directory.

Please consider adding the proper tag for the Karpenter subnet autodiscovery. We usually associate these tags with the private AWS Subnets:

tags = merge(local.base_tags, {
    Tier                                              = "Private"
    "kubernetes.io/role/internal-elb"                 = "1"
    "kubernetes.io/cluster/<Your EKS cluster name>"   = "shared"
    "karpenter.sh/discovery"                          = "<Your EKS cluster name>"
  })

Unless your AWS account has already been onboarded to EC2 Spot, you will need to create the service-linked role to avoid ServiceLinkedRoleCreationNotPermitted issue:

aws iam create-service-linked-role --aws-service-name spot.amazonaws.com

For Karpenter installation, please log out of the Amazon ECR Public registry before terraform apply, using this command:

helm registry logout public.ecr.aws
docker logout public.ecr.aws

You can check why these steps are necessary in AWS Doc, Karpenter official manual, Karpenter troubleshooting

"alekc/kubectl" is the only tested source for the kubectl TF provider.

In v1.31.2 karpenter crd installation as separate helm chart was introduced. It was done to handle crd update issue in karpenter future karpenter updates. If you have already installed karpenter with this module, we advising to migrate existed CRDs under TF managing. To do this please follow these steps:

1. Update module version to => 1.31.2 and set this parameters manage_crd = true as in snippet above

module "eks-aux" {
  source = "git@github.com:automat-it/terraform-aws-eks-auxiliary.git?ref=v1.31.2"
  module "secure-eks" {
    }
    karpenter = {
      enabled                      = true
      manage_crd                   = true
      ...
    }
    ...
}

2. Update CRDs (EC2NodeClass,NodePool,NodeClaim) in cluster with these metadata

metadata:
  labels:
    app.kubernetes.io/managed-by: Helm
  annotations:
    app.kubernetes.io/managed-by: Helm
    meta.helm.sh/release-name: karpenter-crd
    meta.helm.sh/release-namespace: general

3. Run terraform plan and check drift. It will propose you to install new helm chart called karpenter-crd.
4. Run terraform apply. As a result TF will start manage old CRDs under new entity in state that we created and at the same time no changes would done on infra. After all steps will be done you are free to update karpenter version using this module, CRDs will be updated automatically

To add a service to the module, follow these steps:

1. Place the component file for new service in the root directory, e.g. external-dns.tf.
2. Ensure that any required variables for the new component are added to the services variable in variables.tf file, like:

variable "services" {
  type = object({
    new_service = optional(object({
      enabled                = bool
      helm_version           = optional(string, "version")
      namespace              = optional(string, "namespace")
      nodepool               = optional(string, "nodepool")
      additional_helm_values = optional(string, "")
      }), { enabled = false }),
  })
}

3. Add outputs for new service to outputs.tf file, like:

output "new_service_iam_role_arn" {
  description = "The ARN of the IAM role used by the New service (IAM Roles for Service Accounts)."
  value       = var.services.new_service.enabled ? module.new_service[0].iam_role_arn : null
}

output "new_service_iam_role_id" {
  description = "The ID of the IAM role used by the New service (IAM Roles for Service Accounts)."
  value       = var.services.new_service.enabled ? module.new_service[0].iam_role_id : null
}

4. Thoroughly test your changes to ensure proper functionality.
5. Once you've completed testing, submit a pull request with your changes for review and integration.

Main branch is following Terraform AWS 6.x Provider Version, and latest version of EKS. To work with 5.x terraform provider version please checkout on - main-terraform-aws-v5-provider