Skip to main content

Ingress and DNS

Your applications and users reach a self-managed region through its gateway. This page explains why the gateway needs a wildcard domain, the ways to run a region without your own public wildcard DNS record, and how to keep calls from apps in the same cluster inside the cluster. For the full list of hostnames the gateway serves, see Inbound network connections.

Why the region needs a wildcard domain​

The gateway serves every project in the region on one address and one port. It picks the project from the hostname of each request. Each project gets two hostnames, made from a project ID that Catalyst assigns when the project is created:

http-prj<project-id>.<domain>
grpc-prj<project-id>.<domain>

Anyone with access to the region can create a project from the CLI, the console or the API, at any time, without touching your DNS or your certificates. So the region's DNS and TLS must already cover names that don't exist yet. When you bring your own domain, that takes three things:

  • A wildcard DNS record, *.<domain>, that points at the gateway's load balancer.
  • A certificate for *.<domain>, given to the gateway in gateway.tls.
  • A wildcard ingress on the region, set with diagrid region create --ingress "https://*.<domain>:443".

Calling the gateway by its load balancer address or by the gateway-envoy Service name doesn't work: the hostname matches no project, and the request fails. The region's own hostnames, such as management. and trust., are one label under the same domain, so the wildcard covers them too.

A wildcard matches one label only. A record or certificate for *.example.com doesn't cover http-prj123.catalyst.example.com, so create them for the region's own domain, *.catalyst.example.com.

Run a region without your own public wildcard record​

If your DNS policy doesn't allow a public wildcard record in your domain, you have three options.

Let Diagrid manage the domain​

With a managed domain, Diagrid owns the region's domain, creates its DNS record and issues its certificate. You create neither.

diagrid region create my-region --enable-managed-domain --ingress "https://my-gateway-lb.internal.example.com:443"

Diagrid publishes *.<hash>.r1.privatediagrid.net in public DNS, pointing at the address you give in --ingress: a CNAME for a hostname, an A record for an IP address. Without --ingress, it uses the address of the gateway's load balancer Service. The gateway gets the certificate from the control plane. Quick deployment uses a managed domain by default and configures the data plane for it; if you install with Helm yourself, see the Helm reference for the matching values.

The address can be private. The public record only reveals it, and clients can only reach it from your network. If your policy forbids public records that point at private addresses, use one of the other options.

Create a record for each hostname​

Keep the wildcard ingress on the region, but create individual DNS records instead of a wildcard record:

  1. At install time, create records for the region's fixed hostnames that you use: at least management. and trust., plus any others from Inbound network connections.

  2. After you create each project, read its two hostnames and create a record for each:

    diagrid project create my-project --region my-region
    diagrid project get my-project -o json | jq -r '.status.endpoints.http.url, .status.endpoints.grpc.url'

A project can't be reached until its records exist, so create them in the same pipeline that creates the project.

The certificate must still cover every hostname. A wildcard certificate doesn't need a wildcard DNS record: cert-manager issues *.<domain> with a DNS-01 challenge, which needs only a TXT record at _acme-challenge.<domain>. If you can't use a wildcard certificate, list every hostname in the certificate and reissue it with the new names before anyone uses a new project.

Resolve the domain only inside your network​

Put the wildcard record in DNS that only your network can see, so nothing is published:

  • A private DNS zone, such as an Amazon Route 53 private hosted zone, an Azure Private DNS zone or your internal DNS server, with *.<domain> pointing at the gateway's internal load balancer.
  • The cluster's CoreDNS, for apps that run in the same cluster as the region. See the next section.

The gateway still needs a certificate for *.<domain>. If a private CA signs it, follow Private gateway CA.

Resolve project hostnames to the gateway with CoreDNS​

An app in the same cluster as the region can call its project without leaving the cluster. It still calls the project's hostname, because that is how the gateway finds the project. What changes is where the hostname resolves: to the gateway-envoy Service in cra-agent, not to the load balancer. A CoreDNS rewrite does that for every project in the region and every pod in the cluster, with one rule.

Choose the port​

Inside the cluster, callers reach the gateway on the Service's ports, not the load balancer's. Both are set under gateway.envoy.service in your Helm values:

ValueCarries
port (default 8080)Plain HTTP and gRPC.
httpsPort (443 in the install guides)HTTPS and gRPC over TLS, with the gateway's certificate. Only when gateway.tls is set.

The rewrite applies to every pod, including any Dapr sidecars in this cluster that run outside the region and call sentry., xrs. and the dapr-* hostnames on the region's ingress port. Use it only when httpsPort equals that port, as it does in the install guides (443). Otherwise those callers stop connecting. Set httpsPort to match, or use host aliases, which affect only the pods you set them on.

With httpsPort: 443, apps keep the project endpoints that diagrid project get prints. To skip TLS inside the cluster, use the plain port instead:

DAPR_HTTP_ENDPOINT=http://http-prj<project-id>.<domain>:8080
DAPR_GRPC_ENDPOINT=http://grpc-prj<project-id>.<domain>:8080

Add the rewrite rule​

Adapt this rule: replace the domain, escape its dots, and keep the trailing \.$. answer auto needs CoreDNS 1.8.4 or later.

rewrite stop {
  name regex .*\.catalyst\.example\.com\.$ gateway-envoy.cra-agent.svc.cluster.local. answer auto
}

If the Corefile already has a rewrite that matches the region's domain, replace it with this rule instead of adding this one next to it. Pods look up the name with the cluster's search suffixes first, such as http-prj123.catalyst.example.com.svc.cluster.local, and a rule that matches those names answers before this one.

Don't use the two-line pattern from the CoreDNS examples, a name regex line followed by answer name. It only works when the target keeps part of the original name. Here every hostname maps to the one gateway name, so CoreDNS answers under the gateway's name instead of the one the app asked for. Apps built on glibc then fail to resolve the name. answer auto writes the requested name back into the answer.

Where the rule goes depends on the cluster:

  • AKS. Don't edit the coredns ConfigMap, because AKS resets it. Put the rule in the coredns-custom ConfigMap in kube-system, under a key that ends in .override, then restart CoreDNS with kubectl -n kube-system rollout restart deployment coredns.
  • EKS with the CoreDNS add-on. Add the rule to the Corefile through the add-on's configuration. An edit to the coredns ConfigMap can be overwritten when the add-on updates.
  • GKE with kube-dns, OpenShift and EKS Auto Mode. You can't add a rewrite. Use host aliases.
  • Any other cluster that runs CoreDNS. Add the rule to the server block for . in the coredns ConfigMap in kube-system.

Check the CoreDNS version first. After CoreDNS restarts, check the rule from one of your app's pods. The second command prints the cluster IP of the gateway-envoy Service:

kubectl -n kube-system get deploy coredns -o jsonpath='{.spec.template.spec.containers[0].image}'
kubectl exec <app-pod> -- getent hosts http-prj<project-id>.<domain>

Use host aliases instead​

Host aliases need nothing from DNS, and they change only the pods you set them on. Get the gateway's cluster IP:

kubectl get svc -n cra-agent gateway-envoy -o jsonpath='{.spec.clusterIP}'

Then add the project's two hostnames to your pod spec:

spec:
  hostAliases:
    - ip: "<gateway cluster IP>"
      hostnames:
        - http-prj<project-id>.<domain>
        - grpc-prj<project-id>.<domain>

The cluster IP stays the same while the Service exists. It changes if the Service is deleted and created again, for example when you uninstall and reinstall the region.

Allow the traffic​

Your app runs in your own namespace, so the region's network policies don't apply to it. If you restrict egress from that namespace, allow traffic to the cra-agent namespace. Some network plugins, Cilium among them, don't apply IP-based rules to traffic between pods. On those, also allow ingress to the pods labelled app.kubernetes.io/component: gateway-envoy in cra-agent from your app's namespace.

Last updated on