Skip to main content

AWS Deployment

This guide walks through deploying a Catalyst Enterprise Self-Hosted region into a private AWS VPC. It provisions an EKS cluster, RDS PostgreSQL, and a Bastion host, then installs the Catalyst Helm chart against the resulting cluster.

Demonstration only

This guide is intended as a reference walkthrough for demonstration and proof-of-concept deployments. It is not a production-ready deployment recipe — use the Production Planning guidance and adapt this Terraform to your organization's networking, security, and operational requirements before going to production.

Operating system

The commands in this guide are written for Linux or macOS. Adaptations may be required on Windows.

Architecture overview​

The Terraform in this guide provisions a private VPC and deploys the Catalyst data plane to an EKS cluster inside it. Diagrid Cloud manages the Catalyst control plane over an mTLS connection from inside the VPC.

Prerequisites​

  • Diagrid CLI
  • AWS CLI, configured with credentials for your AWS account
  • Helm
  • jq
  • An AWS account with permissions to create VPC, EKS, EC2, IAM, and related resources

The Terraform that provisions the AWS infrastructure used below is available in the guides/aws directory of the diagridio/charts repository. Clone the repository and run make from inside that directory:

git clone https://github.com/diagridio/charts.git
cd charts/guides/aws

1. Create a Catalyst region​

Use the Diagrid CLI to register a new region. The wildcard ingress domain is configured later, once the Catalyst gateway Load Balancer is available.

diagrid login

# Create a new region (the ingress domain is updated later)
export JOIN_TOKEN=$(diagrid region create my-aws-region \
  --ingress placeholder.example.com | jq -r .joinToken)

# Create an API key the Diagrid CLI can use later from inside the VPC
export API_KEY=$(diagrid apikey create \
  --name aws-key \
  --role cra.diagrid:editor \
  --duration 8640 | jq -r .token)

2. Deploy AWS resources​

Use the provided Terraform to deploy the required infrastructure to your AWS account. This provisions an EKS cluster in a private VPC alongside a Bastion host that you can use to securely access the cluster for admin operations.

# Authenticate with AWS and confirm the active profile
aws sts get-caller-identity

# Initialize Terraform
make init

# Show the Terraform plan
make plan

# Apply the Terraform plan
make apply

3. Connect to the Bastion host​

Connect to the Bastion host via EC2 Instance Connect:

EC2_CONNECT_CMD=$(make output bastion_ec2_instance_connect_command)

# Execute the EC2 connect command
eval $EC2_CONNECT_CMD

The Bastion host resides within the private VPC and can communicate with the EKS cluster.

4. Configure Kubernetes access​

The Bastion host has kubectl pre-installed. Configure it to talk to your EKS cluster.

On your local machine, read the EKS cluster name and region from the Terraform outputs:

make output eks_cluster_name
make output eks_cluster_region

On the Bastion host SSH session, configure kubectl and set the default storage class:

export EKS_CLUSTER_NAME="<value-from-eks_cluster_name-output>"
export EKS_CLUSTER_REGION="<value-from-eks_cluster_region-output>"

aws eks update-kubeconfig --name $EKS_CLUSTER_NAME --region $EKS_CLUSTER_REGION

kubectl config current-context

# Set the default storage class to gp2
kubectl patch storageclass gp2 \
  -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Optional: Install the krew kubectl plugin manager​

Install krew on the Bastion host for quality-of-life utilities like kubectx and kubens:

sudo yum install git --assumeyes
(
  set -x; cd "$(mktemp -d)" &&
  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
  KREW="krew-${OS}_${ARCH}" &&
  curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
  tar zxvf "${KREW}.tar.gz" &&
  ./"${KREW}" install krew
)

cat >> ~/.bashrc << EOF
export PATH="\${KREW_ROOT:-\$HOME/.krew}/bin:\$PATH"
EOF

Restart your shell so the PATH change takes effect, then install kubectx, kubens, and a small alias file:

kubectl krew install ctx
kubectl krew install ns

cat >> ~/.bashrc << EOF
source ~/.alias
EOF

cat > ~/.alias << EOF
alias k='kubectl'
alias kp='kubectl get pods'
alias kpa='kubectl get pods --all-namespaces'
alias kf='kubectl logs -f'
alias kx='k ctx'
alias kns='k ns'
EOF

5. Install the AWS Load Balancer controller​

On your local machine, read the cluster name and the controller IAM role ARN from the Terraform outputs:

make output eks_cluster_name
make output aws_load_balancer_controller_role_arn

On the Bastion host SSH session, install the controller via Helm:

helm repo add eks https://aws.github.io/eks-charts
helm repo update eks

helm upgrade --install aws-load-balancer-controller eks/aws-load-balancer-controller \
    --namespace kube-system \
    --set clusterName=<value-from-eks_cluster_name> \
    --set serviceAccount.create=true \
    --set serviceAccount.name=aws-load-balancer-controller \
    --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=<value-from-aws_load_balancer_controller_role_arn-output>

6. Install cert-manager​

On your local machine, read the cert-manager IAM role ARN from the Terraform outputs:

make output cert_manager_role_arn

On the Bastion host SSH session, install cert-manager via Helm:

helm repo add jetstack https://charts.jetstack.io --force-update

helm upgrade --install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version 1.42.0-rc.2 \
  --set crds.enabled=true \
  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=<value-from-cert_manager_role_arn-output>

Install a Let's Encrypt ClusterIssuer​

Create a ClusterIssuer that uses the DNS-01 challenge via Route53. Pass the cert-manager IAM role ARN read above:

cat >> ~/lets-encrypt-cluster-issuer.yaml << EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
  namespace: cert-manager
spec:
  acme:
    email: support@diagrid.io
    privateKeySecretRef:
      name: letsencrypt
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
    - dns01:
        route53:
          region: us-west-2
          role: <value-from-cert_manager_role_arn-output>
          auth:
            kubernetes:
              serviceAccountRef:
                name: cert-manager
EOF

kubectl apply -f ~/lets-encrypt-cluster-issuer.yaml

7. Install monitoring tools​

These are recommended utilities for observing the cluster and Catalyst components. They are not strictly required for Catalyst to function.

Metrics server​

helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server/

helm upgrade --install metrics-server metrics-server/metrics-server

Prometheus node exporter​

cat > ~/prometheus-node-exporter-config.yaml << EOF
podAnnotations:
  prometheus.io/path: /metrics
  prometheus.io/port: "9100"
  prometheus.io/scrape: "true"
rbac:
  pspEnabled: false
resources:
  limits:
    memory: 50Mi
  requests:
    cpu: 10m
    memory: 25Mi
EOF

helm install prometheus-node-exporter \
  oci://ghcr.io/prometheus-community/charts/prometheus-node-exporter \
  --values prometheus-node-exporter-config.yaml

kube-state-metrics​

cat > ~/kube-state-metrics-config.yaml << EOF
podAnnotations:
  prometheus.io/port: "8080"
  prometheus.io/scrape: "true"
EOF

helm install kube-state-metrics \
  oci://ghcr.io/prometheus-community/charts/kube-state-metrics \
  --values kube-state-metrics-config.yaml

8. Configure and install Catalyst​

Secrets backend

This example uses the default Kubernetes Secrets backend. Catalyst can also be configured to use AWS Secrets Manager — see the Helm Chart Reference for the full set of options.

Read the RDS PostgreSQL connection details from the Terraform outputs:

make output postgresql_endpoint
make output postgresql_master_user_secret_arn
make output eks_cluster_region

# Fetch the master user password from AWS Secrets Manager
aws --region $(make output eks_cluster_region) secretsmanager get-secret-value \
  --secret-id $(make output postgresql_master_user_secret_arn) \
  --query SecretString --output text | jq '.password'

On the Bastion host SSH session, create a Helm values file that wires Catalyst up to RDS and exposes the gateway via an AWS Network Load Balancer (NLB):

export RDS_POSTGRESQL_ENDPOINT="<value-from-postgresql_endpoint-output>"
export RDS_POSTGRESQL_PASSWORD="<value-from-aws-cli>"

cat > catalyst-values.yaml << EOF
agent:
  config:
    project:
      default_managed_state_store_type: postgresql-shared-external
      external_postgresql:
        enabled: true
        auth_type: connectionString
        namespace: postgresql
        connection_string_host: $RDS_POSTGRESQL_ENDPOINT
        connection_string_port: 5432
        connection_string_username: postgres
        connection_string_password: "$RDS_POSTGRESQL_PASSWORD"
        connection_string_database: catalyst
gateway:
  tls:
    enabled: true
    secretName: "cert-wildcard"
  envoy:
    service:
      type: LoadBalancer
      httpsPort: 443
      httpsTargetPort: 8443
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: "tcp"
        service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
        service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "instance"
EOF

helm install catalyst oci://public.ecr.aws/diagrid/catalyst \
     -n cra-agent \
     --create-namespace \
     -f catalyst-values.yaml \
     --set join_token="${JOIN_TOKEN}" \
     --version 1.53.0

9. Configure ingress and a wildcard TLS certificate​

Create a Route53 hosted zone and wildcard TLS certificate for a domain you own. This allows applications to reach the Catalyst gateway over a friendly DNS name and TLS.

REGION_INGRESS_ENDPOINT="<somename>.<domain that you own>" make apply

make output region_ingress_endpoint

# Update the region with the wildcard ingress domain
diagrid region update my-aws-region \
  --ingress "<value-from-region_ingress_endpoint-output>"

Delegate the subdomain to Route53 by adding NS records in your domain registrar's DNS settings that point to the Route53 hosted zone's nameservers. Once delegation propagates, dig should resolve the NLB:

dig whatever.<somename>.<domain that you own>

Issue a Let's Encrypt wildcard certificate​

On your local machine:

make output region_wildcard_domain

On the Bastion host SSH session:

cat >> ~/wildcard-certificate.yaml << EOF
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: cert-wildcard
  namespace: cra-agent
spec:
  dnsNames:
  - "<value-from-region_wildcard_domain-output>"
  issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    name: letsencrypt
  secretName: cert-wildcard
  usages:
  - digital signature
  - key encipherment
EOF

kubectl apply -f ~/wildcard-certificate.yaml

10. Verify the installation​

Wait for all Catalyst pods to become ready. This can take several minutes.

kubectl -n cra-agent wait --for=condition=ready pod --all --timeout=5m

11. Create a project and deploy App IDs​

On your local machine, create a project in the new region and a couple of App IDs to test connectivity:

diagrid project create aws-project --region my-aws-region --use

diagrid appid create app1
diagrid appid create app2

# Wait until the App IDs are ready
diagrid appid list

Because the Catalyst gateway is internal to the VPC, test invocation from within the VPC. Open a new terminal window on your local machine and connect to the Bastion host again to start a listener:

EC2_CONNECT_CMD=$(make output bastion_ec2_instance_connect_command)
eval $EC2_CONNECT_CMD

# On the new Bastion host SSH session
diagrid login --api-key="$API_KEY"
diagrid project use aws-project

# The Diagrid CLI can create a listener for an App ID and print any requests
# sent to it. Wait until you see:
# ✅ Connected App ID "app1" to http://localhost:<port> ⚡️
diagrid listen -a app1

From the original Bastion session, send a service-invocation request between App IDs:

# Call app1 from app2 via the internal gateway using the wildcard domain
diagrid call invoke get app1.hello -a app2

You should see the request received ('method': 'GET', 'url': '/hello') on the listener. This proves that you can use Dapr's service invocation API by calling an App ID via the internal NLB using the Catalyst region's wildcard domain.

To open the Catalyst web console for the project:

diagrid web

12. Write your applications​

Now that the Catalyst region is up and a project with two App IDs is in place, head over to the Connect to Catalyst guide to start building applications that use these App IDs. Applications running inside the VPC can reach the internal gateway directly at http://<GATEWAY_HOSTNAME>:8080.

For applications running outside the VPC, you must own a domain that can route traffic to your Catalyst gateway, configure TLS on the gateway (set gateway.tls.enabled: true and create a Kubernetes TLS secret named gateway-tls-certs in the cra-agent namespace), and ensure your applications trust the root CA for the certificate you use. Then point your applications at the Catalyst region with the standard Dapr environment variables:

  • DAPR_GRPC_ENDPOINT=<your-project-grpc-url>
  • DAPR_HTTP_ENDPOINT=<your-project-http-url>
  • DAPR_API_TOKEN=<your-appid-api-token>

Authenticate AWS components with IAM Roles Anywhere​

Catalyst App IDs can authenticate to AWS services — such as DynamoDB, S3, Secrets Manager, SNS/SQS, or RDS — using AWS IAM Roles Anywhere instead of storing long-lived AWS access keys in component metadata. Each App ID already has a SPIFFE X.509 identity issued by the region's Dapr Sentry; IAM Roles Anywhere lets that certificate be exchanged for short-lived AWS credentials at runtime.

Prerequisite

You need the AWS CLI configured with permission to manage IAM and Roles Anywhere resources, the Diagrid CLI, and an App ID in a Catalyst project. The examples also use jq.

How it works​

  • Each App ID is issued a SPIFFE X.509 SVID by the region's Dapr Sentry, of the form spiffe://<trust-domain>/ns/prj-<project-uid>/<appid>.
  • You register the region's certificate authority (CA) as a Roles Anywhere trust anchor in your AWS account, so AWS trusts certificates issued by Sentry.
  • A Roles Anywhere profile links the trust anchor to an IAM role.
  • The role's trust policy allows the rolesanywhere.amazonaws.com service principal to assume it only for that App ID's SPIFFE ID, matched on the aws:PrincipalTag/x509SAN/URI condition. This is what binds a specific App ID to a specific AWS role.
  • At runtime the App ID's sidecar presents its SVID to IAM Roles Anywhere (CreateSession) and exchanges it for temporary STS credentials. No static keys are stored anywhere.

The setup-catalyst-iamra.sh helper script in guides/aws automates the trust anchor, role, and profile for a single App ID. Run it with --project and --appid, or follow the manual steps below.

./setup-catalyst-iamra.sh --project <your-project> --appid <your-app-id>

It resolves the App ID's SPIFFE ID, registers the region's CA as a trust anchor, creates the IAM role with the correct trust policy, and creates the profile — then prints the three ARNs you wire into the component.

Set up IAM Roles Anywhere manually​

The following steps reproduce the helper script for one App ID. The trust anchor (step 3) is created once per region; repeat steps 4–5 for each App ID that needs its own role.

1. Set environment variables​

export CATALYST_PROJECT="aws-project"
export CATALYST_APP="app1"
export AWS_REGION="eu-west-1"

2. Resolve the App ID's SPIFFE ID​

The role's trust policy is scoped to this exact value:

SPIFFE_ID=$(diagrid appid get "$CATALYST_APP" --project "$CATALYST_PROJECT" -o json | jq -r '.status.spiffeId')
echo "$SPIFFE_ID"
# spiffe://<trust-domain>/ns/prj-<project-uid>/app1

3. Register the trust anchor​

Each region serves its CA bundle from a trust endpoint derived from the region's ingress. Resolve that endpoint, fetch the bundle, and register it as a Roles Anywhere trust anchor:

# Resolve the region for your project, then the region's ingress endpoint
CATALYST_REGION=$(diagrid project get "$CATALYST_PROJECT" -o json | jq -r '.spec.region')
INGRESS=$(diagrid region get "$CATALYST_REGION" -o json | jq -r '.spec.ingress')
# INGRESS is of the form https://*.<wildcard-domain>:<port>

# The trust (CA) endpoint replaces the "*." wildcard host with "trust."
TRUST_ENDPOINT=$(echo "$INGRESS" | sed -E 's#//\*\.#//trust.#')

# Fetch the region's CA bundle and register it as the trust anchor
curl -fsS "$TRUST_ENDPOINT" -o diagrid-ca.pem

jq -n --rawfile pem diagrid-ca.pem \
  '{name:"catalyst", source:{sourceType:"CERTIFICATE_BUNDLE", sourceData:{x509CertificateData:$pem}}, enabled:true}' \
  > trust-anchor.json

TRUST_ANCHOR_ARN=$(aws rolesanywhere create-trust-anchor --region "$AWS_REGION" \
  --cli-input-json file://trust-anchor.json \
  --query 'trustAnchor.trustAnchorArn' --output text)

4. Create the IAM role and trust policy​

The trust policy lets Roles Anywhere assume the role only for this App ID's SPIFFE ID:

cat > role-trust.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "Service": "rolesanywhere.amazonaws.com" },
    "Action": ["sts:AssumeRole", "sts:TagSession", "sts:SetSourceIdentity"],
    "Condition": {
      "ArnEquals": { "aws:SourceArn": "${TRUST_ANCHOR_ARN}" },
      "StringEquals": { "aws:PrincipalTag/x509SAN/URI": "${SPIFFE_ID}" }
    }
  }]
}
EOF

ROLE_ARN=$(aws iam create-role \
  --role-name "catalyst-${CATALYST_APP}" \
  --assume-role-policy-document file://role-trust.json \
  --query 'Role.Arn' --output text)

5. Create the Roles Anywhere profile​

PROFILE_ARN=$(aws rolesanywhere create-profile --region "$AWS_REGION" \
  --name "catalyst-${CATALYST_APP}" \
  --role-arns "$ROLE_ARN" \
  --enabled \
  --query 'profile.profileArn' --output text)

You now have the three ARNs every IAM Roles Anywhere component needs: TRUST_ANCHOR_ARN, PROFILE_ARN, and ROLE_ARN. The role grants no access yet — the example below attaches a permissions policy.

Example: AWS DynamoDB state store​

This wires the App ID up to a DynamoDB table as a Dapr state store.

1. Create the DynamoDB table​

The Dapr DynamoDB state store requires a partition key named key:

aws dynamodb create-table --region "$AWS_REGION" \
  --table-name Contracts \
  --attribute-definitions AttributeName=key,AttributeType=S \
  --key-schema AttributeName=key,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST
Partition key

The partition key must be named key (type S), with no sort key — this is what the Dapr DynamoDB state store expects. To use a different name, set the component's partitionKey metadata field.

2. Grant the role access to the table​

Attach a read/write permissions policy to the role created above (replace <account-id>):

cat > perms.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "dynamodb:GetItem", "dynamodb:BatchGetItem", "dynamodb:Query", "dynamodb:Scan",
      "dynamodb:PutItem", "dynamodb:UpdateItem", "dynamodb:DeleteItem", "dynamodb:BatchWriteItem",
      "dynamodb:TransactGetItems", "dynamodb:TransactWriteItems"
    ],
    "Resource": [
      "arn:aws:dynamodb:${AWS_REGION}:<account-id>:table/Contracts",
      "arn:aws:dynamodb:${AWS_REGION}:<account-id>:table/Contracts/index/*"
    ]
  }]
}
EOF

aws iam put-role-policy --role-name "catalyst-${CATALYST_APP}" \
  --policy-name catalyst-access --policy-document file://perms.json

3. Create the Catalyst component​

Create the component with the AWS: IAM Roles Anywhere authentication profile — the three ARNs from the setup steps — and scope it to the App ID:

diagrid component create dynamodb \
  --type state.aws.dynamodb \
  --project "$CATALYST_PROJECT" \
  --metadata region="$AWS_REGION" \
  --metadata table=Contracts \
  --metadata assumeRoleArn="$ROLE_ARN" \
  --metadata trustAnchorArn="$TRUST_ANCHOR_ARN" \
  --metadata trustProfileArn="$PROFILE_ARN" \
  --scopes "$CATALYST_APP" \
  --wait

Once the component is ready, the App ID can read and write state through the Dapr state API, and Catalyst authenticates every DynamoDB call with short-lived credentials minted from the App ID's identity. The same assumeRoleArn / trustAnchorArn / trustProfileArn profile is available on the other AWS components (Secrets Manager, SNS/SQS, S3, RDS/PostgreSQL, and more) — swap the --type and the IAM permissions policy for the target service.

Last updated on