This guide shows how to deploy own Docker apps (with AdminForth as example) to Amazon EC2 instance with K3s and Terraform involving pushing images into Amazon ECR.

Needed resources:

• AWS account where we will auto-spawn EC2 instance. We will use t3a.small instance (2 vCPUs, 2GB RAM) which costs ~14$ per month in us-west-2 region (cheapest region). Also it will take $2 per month for EBS gp2 storage (20GB) for EC2 instance.
• Also AWS ECR will charge for $0.09 per GB of data egress traffic (from EC2 to the internet) - this needed to load docker build cache.

The setup shape:

• Build is done using IaaC approach with HashiCorp Terraform, so almoast no manual actions are needed from you. Every resource including EC2 server instance is described in code which is commited to repo.
• Docker images and build cache are stored on Amazon ECR
• Total build time for average commit to AdminForth app (with Vite rebuilds) is around 3 minutes.

Why exactly K3s?

Previously, our blog featured posts about different types of application deployment, but without the use of Kubernetes. This post will look at the cheapest option for deploying an application using k3s (a lightweight version of k8s Kubernetes). This option is more interesting than most of the alternatives, primarily because of its automation and scalability (it is, of course, inferior to the “older” K8s, but it also requires significantly fewer resources).

How we will store containers?

The ECR repository will be used for storage. Since we are working with AWS, this is the most reliable option. The image must be assembled from local files on the machine and then sent to the Amazon server. All instructions for performing these actions will be provided below.

Prerequisites

I will assume you run Ubuntu (Native or WSL2).

You should have terraform, here is official repository:

wget -O - https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

AWS CLI:

sudo snap install aws-cli --classic

HELM:

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

HELMFILE and Helm-Diff plugin:

wget -O helmfile_linux_amd64.tar.gz https://github.com/helmfile/helmfile/releases/download/v0.162.0/helmfile_0.162.0_linux_amd64.tar.gz
tar -zxvf helmfile_linux_amd64.tar.gz
sudo mv helmfile /usr/local/bin/helmfile
rm helmfile_linux_amd64.tar.gz

helm plugin install https://github.com/databus23/helm-diff

Also you need Doker Daemon running. We recommend Docker Desktop running. ON WSL2 make sure you have Docker Desktop WSL2 integration enabled.

docker version

It is also worth having kubectl locally on your machine for more convenient interaction with nodes and pods, but this is not mandatory.

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

Last step is download ansible:

sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible -y

Practice - deploy setup

Assume you have your AdminForth project in myadmin.

Step 1 - create a SSH keypair

Make sure you are still in deploy folder, run next command:

deploy

mkdir .keys && ssh-keygen -f .keys/id_rsa -N ""

Now it should create deploy/.keys/id_rsa and deploy/.keys/id_rsa.pub files with your SSH keypair. Terraform script will put the public key to the EC2 instance and will use private key to connect to the instance. Also you will be able to use it to connect to the instance manually.

Step 2 - .gitignore file

Create deploy/.gitignore file with next content:

.terraform/
.keys/
*.tfstate
*.tfstate.*
*.tfvars
tfplan
session-manager-plugin.deb
.terraform.lock.hcl

Step 3 - Terraform folder

First of all install Terraform as described here terraform installation.

After this create folder ../deploy/terraform

Create file main.tf in deploy/terraform folder:

deploy/terraform/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

}

locals {
  aws_region           = "us-west-2"
  vpc_cidr             = "10.0.0.0/16"
  subnet_a_cidr        = "10.0.10.0/24"
  subnet_b_cidr        = "10.0.11.0/24"
  az_a                 = "us-west-2a"
  az_b                 = "us-west-2b"
  app_name             = <your_app_name>
  app_source_code_path = "../../"
  ansible_dir          = "../ansible/playbooks"
  app_files            = fileset(local.app_source_code_path, "**")

  image_tag = sha256(join("", [
    for f in local.app_files :
    try(filesha256("${local.app_source_code_path}/${f}"), "")
    if length(regexall("^deploy/", f)) == 0
    && length(regexall("^\\.vscode/", f)) == 0
    && length(regexall("^node_modules/", f)) == 0
    && length(regexall("^\\.gitignore", f)) == 0
  ]))

  ingress_ports = [
    { from = 22, to = 22, protocol = "tcp", desc = "SSH" },
    { from = 80, to = 80, protocol = "tcp", desc = "App HTTP (Traefik)" },
    { from = 443, to = 443, protocol = "tcp", desc = "App HTTPS (Traefik)" },
    { from = 6443, to = 6443, protocol = "tcp", desc = "Kubernetes API" }
  ]
}

provider "aws" {
  region  = local.aws_region
  profile = "myaws"
}

data "aws_ami" "ubuntu_22_04" {
  most_recent = true
  owners      = ["099720109477"] # Canonical ubuntu account ID

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }
}

resource "aws_key_pair" "app_deployer" {
  key_name   = "terraform-deploy_${local.app_name}-key"
  public_key = file("../.keys/id_rsa.pub") # Path to your public SSH key
}

resource "aws_instance" "ec2_instance" {
  instance_type = "t3a.small"
  ami           = data.aws_ami.ubuntu_22_04.id

  iam_instance_profile = aws_iam_instance_profile.instance_profile.name

  subnet_id                   = aws_subnet.public_a.id
  vpc_security_group_ids      = [aws_security_group.app_sg.id]
  associate_public_ip_address = true
  key_name                    = aws_key_pair.app_deployer.key_name

  tags = {
    Name = local.app_name
  }

  # prevent accidental termination of ec2 instance and data loss
  lifecycle {
    create_before_destroy = true #uncomment in production
    #prevent_destroy       = true       #uncomment in production
    ignore_changes = [ami]
  }

  root_block_device {
    volume_size = 20 // Size in GB for root partition
    volume_type = "gp2"

    # Even if the instance is terminated, the volume will not be deleted, delete it manually if needed
    delete_on_termination = true #change to false in production if data persistence is needed
  }

}

resource "local_file" "ansible_inventory" {
  content = <<EOF
[k3s_nodes]
${aws_instance.ec2_instance.public_ip} ansible_user=ubuntu ansible_ssh_private_key_file=.keys/id_rsa
EOF

  filename = "../ansible/inventory.ini"
}

👆 Replace <your_app_name> with your app name (no spaces, only underscores or letters)

We will also need a file container.tf

deploy/terraform/container.tf

resource "aws_ecr_repository" "app_repo" {
  name = local.app_name

  image_tag_mutability = "MUTABLE"
  image_scanning_configuration {
    scan_on_push = true
  }
  force_delete = true
}

data "aws_caller_identity" "current" {}

resource "local_file" "image_tag" {
  content    = local.image_tag
  filename   = "${path.module}/image_tag.txt"
}

This file contains a script that builds the Docker image locally. This is done for more flexible deployment. When changing the program code, there is no need to manually update the image on EC2 or in the repository. It is updated automatically with each terraform apply. Below is a table showing the time it takes to build this image from scratch and with minimal changes.

Feature	Time
Initial build time*	0m45.445s
Rebuild time (changed index.ts)*	0m26.757s

_{* All tests done from local machine (Intel(R) Core(TM) i7 9760H, Docker Desktop/Ubuntu 32 GB RAM, 300Mbps up/down) up to working state}

Also, resvpc.tf

deploy/terraform/resvpc.tf

resource "aws_vpc" "main" {
  cidr_block = local.vpc_cidr

  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = { Name = "main-vpc" }
}

resource "aws_subnet" "public_a" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = local.subnet_a_cidr
  map_public_ip_on_launch = true
  availability_zone       = local.az_a
  tags = {
    Name = "public-a"
  }
}

resource "aws_subnet" "public_b" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = local.subnet_b_cidr
  map_public_ip_on_launch = true
  availability_zone       = local.az_b
  tags = {
    Name = "public-b"
  }
}

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.main.id
  tags   = { Name = "main-igw" }
}

resource "aws_route_table" "public_rt" {
  vpc_id = aws_vpc.main.id
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }
  tags = { Name = "public-rt" }
}

resource "aws_route_table_association" "public_a_assoc" {
  subnet_id      = aws_subnet.public_a.id
  route_table_id = aws_route_table.public_rt.id
}

resource "aws_route_table_association" "public_b_assoc" {
  subnet_id      = aws_subnet.public_b.id
  route_table_id = aws_route_table.public_rt.id
}

resource "aws_security_group" "app_sg" {
  name   = "${local.app_name}-SecurityGroup"
  vpc_id = aws_vpc.main.id

  dynamic "ingress" {
    for_each = local.ingress_ports
    content {
      from_port   = ingress.value.from
      to_port     = ingress.value.to
      protocol    = ingress.value.protocol
      cidr_blocks = ["0.0.0.0/0"]
      description = ingress.value.desc
    }
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_iam_role" "node_role" {
  name = "${local.app_name}node-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "ec2.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy_attachment" "ecr_read_only_attach" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.node_role.name
}

resource "aws_iam_role_policy_attachment" "ssm_core_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  role       = aws_iam_role.node_role.name
}

resource "aws_iam_instance_profile" "instance_profile" {
  name = "${local.app_name}-instance-profile"
  role = aws_iam_role.node_role.name
}

And outputs.tf

deploy/terraform/outputs.tf

output "app_endpoint" {
  value = "http://${aws_instance.ec2_instance.public_dns}"
}

output "ssh_connect_command" {
  value = "ssh -i .keys/id_rsa ubuntu@${aws_instance.ec2_instance.public_dns}"
}

output "hash" {
  value = local.image_tag
}

output "ecr_repository_url" {
  value = aws_ecr_repository.app_repo.repository_url
}

output "account_id" {
  value = data.aws_caller_identity.current.account_id
}

output "aws_region" {
  value = local.aws_region
}

output "public_ip" {
  value = aws_instance.ec2_instance.public_ip
}

Step 4 - Helm

Helm is a command-line tool and a set of libraries that helps manage applications in Kubernetes.

Helm Chart (Chart) is a package containing everything needed to run an application in Kubernetes. It's the equivalent of apt or yum packages in Linux.

A chart has this structure:

helm_charts/
├── Chart.yaml        # Metadata about the chart (name, version)
├── values.yaml       # Default values (configuration)
└── templates/        # Folder with Kubernetes templates (YAML files)
    ├── deployment.yaml
    ├── service.yaml
    ├── ingress.yaml
    └── ...

Step 5 - Provider Helm and Helmfile

Now we need to create a deploy/helm folder.

You need to create a file Chart.yaml in it:

deploy/helm/Chart.yaml

apiVersion: v2
name: myadmink3s # SET YOUR APP NAME
description: Helm chart for myadmin app
version: 0.1.0
appVersion: "1.0.0"

And values.yaml:

deploy/helm/values.yaml

appName: myadmink3s # SET YOUR APP NAME LIKE IN Chart.yaml
appNameSpace: myadmin # SET YOUR APP NAMESPACE
containerPort: 3500
servicePort: 80
adminSecret: "your_secret"
ecrImageFull: ""

And finally, helmfile.yaml. Helmfile allows us to dynamically inject environment variables and manage the chart installation declaratively:

deploy/helm/helmfile.yaml

releases:
  - name: '{{ requiredEnv "APP_NAMESPACE" }}'
    namespace: '{{ requiredEnv "APP_NAMESPACE" }}'
    createNamespace: true
    chart: .
    values:
      - values.yaml
      - ecrImageFull: '{{ requiredEnv "IMAGE_FULL_TAG" }}'

After this create a deploy/helm/templates folder:

And create files here:

deployment.yaml

deploy/helm/templates/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.appName }}-deployment
  namespace: {{ .Values.appNameSpace }}
spec:
  replicas: 1
  selector:
    matchLabels:
      app: {{ .Values.appName }}
  template:
    metadata:
      labels:
        app: {{ .Values.appName }}
    spec:
      containers:
      - name: {{ .Values.appName }}
        image: "{{ .Values.ecrImageFull }}" 
        ports:
        - containerPort: {{ .Values.containerPort }}
        env:
        - name: "ADMINFORTH_SECRET"
          value: "{{ .Values.adminSecret }}"

ingress.yaml

deploy/helm/templates/ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ .Values.appName }}-ingress
  namespace: {{ .Values.appNameSpace }}
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: {{ .Values.appName }}-service
            port:
              number: {{ .Values.servicePort }}

And service.yaml

deploy/helm/templates/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: {{ .Values.appName }}-service
  namespace: {{ .Values.appNameSpace }}
spec:
  type: ClusterIP
  selector:
    app: {{ .Values.appName }}
  ports:
  - port: {{ .Values.servicePort }}
    targetPort: {{ .Values.containerPort }}

The comments in the values.yaml and Chart.yaml files indicate the names of the variables that need to be replaced. They must correspond to the variables in Ansible, which will be discussed later.

Step 6 - Ansible

If we explain the logic of deployment, Ansible plays a very important role here. If Terraform is used exclusively to configure cloud infrastructure in AWS, Ansible prepares it for the deployment of a Kubernetes cluster. Ansible Playbooks, in simple terms, are templates for configuring the system (of course, their functionality is much broader, but in this deployment method and in most cases, this is how they are used). That is, after preparing the instance with Terraform, it launches Ansible, which prepares it for the deployment of the Kubernetes cluster, after which it launches Helm. This method allows for the highest quality deployment, because each stage is handled by software specialized for that particular stage.

So, create /deploy/ansible/playbooks folder

Then the file playbook.yaml

/deploy/ansible/playbooks/playbook.yaml

---
- name: Deploy application
  hosts: k3s_nodes
  become: true
  vars:
    k3s_version: "v1.27.3+k3s1"

  tasks:

    - name: Read Docker image tag (local)
      ansible.builtin.set_fact:
        image_tag: "{{ lookup('file', '../../terraform/image_tag.txt') }}"
      delegate_to: localhost

    - name: Install unzip
      ansible.builtin.apt:
        name: unzip
        state: present
        update_cache: true

    - name: Download AWS CLI v2
      ansible.builtin.get_url:
        url: "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip"
        dest: /tmp/awscliv2.zip
        mode: '0644'

    - name: Unzip AWS CLI
      ansible.builtin.unarchive:
        src: /tmp/awscliv2.zip
        dest: /tmp
        remote_src: true

    - name: Install AWS CLI
      ansible.builtin.command: /tmp/aws/install --update
      args:
        creates: /usr/local/bin/aws

    - name: Update apt cache
      ansible.builtin.apt:
        update_cache: true

    - name: Install required packages
      ansible.builtin.apt:
        name:
          - curl
          - sudo
          - software-properties-common
          - apt-transport-https
          - ca-certificates
        state: present

    - name: Download k3s installation script
      ansible.builtin.get_url:
        url: https://get.k3s.io
        dest: /tmp/install_k3s.sh
        mode: '0700'

    - name: Install k3s
      ansible.builtin.command: /tmp/install_k3s.sh
      environment:
        INSTALL_K3S_VERSION: "{{ k3s_version }}"
        INSTALL_K3S_EXEC: "--tls-san {{ inventory_hostname }}"
      args:
        creates: /usr/local/bin/k3s

    - name: Get ECR token
      ansible.builtin.command: aws ecr get-login-password --region us-west-2
      register: ecr_token
      changed_when: false

    - name: Configure K3s registry for ECR
      ansible.builtin.copy:
        dest: /etc/rancher/k3s/registries.yaml
        content: |
          configs:
            "735356255780.dkr.ecr.us-west-2.amazonaws.com":
              auth:
                username: AWS
                password: "{{ ecr_token.stdout }}"
        mode: '0600'

    - name: Restart k3s to apply registry changes
      ansible.builtin.systemd:
        name: k3s
        state: restarted
        enabled: true

    - name: Wait for k3s node to be ready
      ansible.builtin.wait_for:
        path: /usr/local/bin/k3s
        state: present
        timeout: 300

    - name: Ensure ~/.kube directory exists
      ansible.builtin.file:
        path: /root/.kube
        state: directory
        mode: '0700'

    - name: Copy k3s kubeconfig
      ansible.builtin.copy:
        remote_src: true
        src: /etc/rancher/k3s/k3s.yaml
        dest: /root/.kube/config
        owner: root
        group: root
        mode: '0600'

    - name: Replace localhost with public IP in kubeconfig
      ansible.builtin.replace:
        path: /root/.kube/config
        regexp: "server: https://127.0.0.1:6443"
        replace: "server: https://{{ inventory_hostname }}:6443"

Notice the INSTALL_K3S_EXEC: "--tls-san {{ inventory_hostname }}". This ensures that K3s generates a valid TLS certificate for your EC2 instance's public IP address, allowing local connections without security errors.

Step 7 - The Deployment Orchestrator Script

Now, let's create a single bash script that will tie all these tools together natively:

1. Runs Terraform to create the instance.
2. Extracts required variables via terraform output.
3. Builds and pushes the Docker image natively.
4. Waits for the instance SSH to be available.
5. Runs Ansible to configure the instance.
6. Downloads the cluster access credentials (kubeconfig) and runs helmfile apply.

Create a file named deploy.sh inside the deploy/ directory:

deploy/deploy.sh

#!/bin/bash
set -e

SCRIPT_DIR="$(cd "$(dirname "$${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"

echo "Starting deployment process..."

# 1. Run Terraform
echo "Running Terraform..."
cd terraform
terraform init
terraform apply -auto-approve

# Get Outputs
REPO_URL=$(terraform output -raw ecr_repository_url)
ACCOUNT_ID=$(terraform output -raw account_id)
REGION=$(terraform output -raw aws_region)
TAG=$(terraform output -raw hash)
PUBLIC_IP=$(terraform output -raw public_ip)
APP_SOURCE_CODE_PATH="../"
cd ..

# 2. Build and Push Docker Image
echo "Building and Pushing Docker Image..."
echo "LOG: Logging in to ECR..."
aws ecr get-login-password --region $${REGION} | docker login --username AWS --password-stdin $${ACCOUNT_ID}.dkr.ecr.$${REGION}.amazonaws.com

echo "LOG: Building Docker image..."
unset DOCKER_HOST
docker build --pull -t $${REPO_URL}:$${TAG} $${APP_SOURCE_CODE_PATH}

echo "LOG: Pushing image to ECR..."
docker push $${REPO_URL}:$${TAG}
echo "LOG: Build and push complete. TAG=$${TAG}"

# 3. Wait for SSH
echo "Waiting for SSH on $${PUBLIC_IP}..."
for i in {1..20}; do
  nc -zv $${PUBLIC_IP} 22 && echo "SSH is ready!" && break
  echo "Retrying in 5 seconds..."
  sleep 5
  if [ $i -eq 20 ]; then
    echo "SSH not ready after 100 seconds. Exiting."
    exit 1
  fi
done

# 4. Run Ansible
echo "Running Ansible..."
cd ansible
ANSIBLE_HOST_KEY_CHECKING=False ansible-galaxy collection install community.kubernetes
ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i inventory.ini playbooks/playbook.yaml
cd ..

# 5. Deploy via Helmfile
echo "Fetching kubeconfig and deploying via Helmfile..."
ssh -o StrictHostKeyChecking=no -i .keys/id_rsa ubuntu@$${PUBLIC_IP} "sudo cat /etc/rancher/k3s/k3s.yaml" > kubeconfig.yaml
sed -i "s/127.0.0.1/$${PUBLIC_IP}/g" kubeconfig.yaml
sed -i 's/certificate-authority-data:.*/insecure-skip-tls-verify: true/g' kubeconfig.yaml
export KUBECONFIG=$(pwd)/kubeconfig.yaml

cd helm
export APP_NAMESPACE="myadmin"  # <<< SET YOUR APP NAMESPACE HERE
export IMAGE_FULL_TAG="$${ACCOUNT_ID}.dkr.ecr.$${REGION}.amazonaws.com/myadmink3s:$${TAG}"  # Change 'myadmink3s' to your app name in 'helmfile.yaml' & 'Chart.yaml' files as well
helmfile apply
cd ..

echo "Deployment complete!"

You need to change the values of APP_NAMESPACE and IMAGE_FULL_TAG in the deploy/deploy.sh script to match your application name and image tag. Note that IMAGE_FULL_TAG is constructed dynamically from the Terraform outputs and the tag, so you only need to change APP_NAMESPACE.

Don't forget to make the script executable:

chmod +x deploy/deploy.sh

Step 8 - Configure AWS Profile

Open or create file ~/.aws/credentials and add (if not already there):

[myaws]
aws_access_key_id = <your_access_key>
aws_secret_access_key = <your_secret_key>

Then use:

aws configure

And configure your AWS credentials.

Step 9 - Run deployment

All deployment-related actions are automated, so no additional actions are required. To deploy the application, you only need to run the deploy.sh orchestrator script and wait a few minutes. After that, you will be able to connect to the web application using the link you will receive in terraform_output. Next, if you wish, you can add GitHub Actions. To do this, follow the instructions in our other post.

cd deploy
./deploy.sh

All done!

Your application is now deployed on Amazon EC2 and available on the Internet.