Skip to main content

Cluster prerequisites

The following guide details the prerequisites for onboarding a cluster to Conductor.

Installation prerequisites​

Before installing Conductor into a cluster, the following requirements must be met:

Supported platforms​

Currently, the following Kubernetes distributions are supported:

  • Amazon Elastic Kubernetes Service (EKS)
  • Google Kubernetes Engine (GKE)
  • Azure Kubernetes Service (AKS)
  • Red Hat OpenShift (v4.x and above)
  • Local Kubernetes i.e. Minikube, kind, and Docker Desktop
Note

Additional Kubernetes distributions have not been tested with Conductor, but you may try them at your own risk.

Cluster resources​

The recommended production resource configurations for the Dapr control plane are defined in the Dapr documentation and should be applied to your Dapr installation. Installing Dapr using Dev or Prod Helm profiles in Conductor set these recommended values for you automatically.

In addition, the cluster resources needed for the Conductor resources are as follows:

  • The Conductor Agent Deployment requests 200m CPU and 400Mi memory and runs two replicas by default.
  • The Conductor Agent Logs Collector DaemonSet uses a modified version of the OpenTelemetry Collector that requests 200m CPU and 400Mi memory per pod.

See Conductor Agent for additional details.

Networking requirements​

Your Kubernetes cluster needs to be online and have outbound connectivity to Diagrid Cloud. The following URLs must be accessible from your clusters:

To run Dapr version upgrades using the upstream Helm chart and container images, you will need the additional URLs to be accessible from your clusters:

Note: If you opt to use Dapr container images from a private registry, it is not necessary to allow-list these addresses. However, it becomes your responsibility to manage the Dapr Helm chart and container images along with any upgrades you want to apply to your clusters. For more details, read the guide on how to use private images.

If you make use of Kubernetes network policies, you need to ensure the Conductor agent can reach the Dapr data plane (Dapr sidecars) in your cluster on port 3501 to access the metadata API.

Required permissions​

To install the Conductor agent on a cluster, a user must have sufficient RBAC permissions - typically, the cluster-admin role.

The Conductor agent itself also requires a set of Kubernetes RBAC permissions to monitor and manage the Dapr control plane and Dapr workloads. These permissions include all those required to run Dapr in a Kubernetes cluster, along with additional permissions needed for installation, upgrades, monitoring and management of Dapr control plane components. See these broken down below.

Dapr control plane​

RBAC rules are defined with resourceNames for the following Kubernetes resources to ensure that the Conductor agent has the least possible permissions to perform its tasks when monitoring and managing the Dapr control plane. Since Conductor manages Dapr worklaods across the whole cluster, some resources need to have cluster-wide permissions.

- apiGroups: ["apps"]
  resources: ["deployments", "daemonsets", "statefulsets"]
  verbs: ["create"]  # create, list and watch verbs cannot use [resourceNames](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-resources)
- apiGroups: ["apps"]
  resources: ["deployments"]
  resourceNames: ["dapr-operator", "dapr-sentry", "dapr-sidecar-injector"]
  verbs: [ "get", "patch", "update"]
- apiGroups: ["apps"]
  resources: ["daemonsets", "deployments"]
  resourceNames: ["diagrid-agent-logs-collector", "diagrid-agent"]
  verbs: [ "get", "patch", "update"]
- apiGroups: ["apps"]
  resources: ["statefulsets"]
  resourceNames: ["dapr-placement-server", "dapr-scheduler-server"]
  verbs: [ "get", "patch", "update", "delete"] # delete is required when going from non-HA to HA mode and when updating a statefulset
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "update","delete"]
  resourceNames: ["dapr-trust-bundle"]
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "update"]
  resourceNames: ["dapr-trust-bundle"]
- apiGroups: ["coordination.k8s.io"]
  resources: ["leases"]
  verbs: ["create"]
- apiGroups: ["coordination.k8s.io"]
  resources: ["leases"]
  verbs: ["get", "update"]
  resourceNames: ["operator.dapr.io", "webhooks.dapr.io"]
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "update"]
  resourceNames: ["operator.dapr.io", "webhooks.dapr.io"]
- apiGroups: ["apps"]
  resources: ["deployments/finalizers"]
  resourceNames: ["dapr-operator", "dapr-sentry", "dapr-sidecar-injector"]
  verbs: ["update"]
- apiGroups: ["apps"]
  resources: ["statefulsets/finalizers"]
  resourceNames: ["dapr-placement-server", "dapr-scheduler-server"]
  verbs: ["update"]
- apiGroups: ["admissionregistration.k8s.io"]
  resources: ["mutatingwebhookconfigurations"]
  verbs: ["create"]
- apiGroups: ["admissionregistration.k8s.io"]
  resources: ["mutatingwebhookconfigurations"]
  verbs: ["get", "patch", "update", "delete"]
  resourceNames: ["dapr-sidecar-injector"]
- apiGroups: ["policy"]
  resources: ["poddisruptionbudgets"]
  verbs: ["create"]
- apiGroups: ["policy"]
  resources: ["poddisruptionbudgets"]
  verbs: ["patch", "update"]
  resourceNames:
    - dapr-operator-disruption-budget
    - dapr-placement-server-disruption-budget
    - dapr-scheduler-server-disruption-budget
    - dapr-sentry-budget
    - dapr-sidecar-injector-disruption-budget
- apiGroups: [""]
  resources: ["services"]
  verbs: [ "get", "update"]
  resourceNames:
    - dapr-placement-server
    - dapr-scheduler-server
    - dapr-sentry
    - dapr-sidecar-injector
    - dapr-api
    - dapr-webhook
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["create"]  # create verbs cannot use [resourceNames](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-resources)
- apiGroups: ["*"]
  resources: ["customresourcedefinitions"]
  verbs: ["get", "patch", "update"]
  resourceNames: ["components.dapr.io", "configurations.dapr.io", "subscriptions.dapr.io", "resiliencies.dapr.io", "httpendpoints.dapr.io", "pluggablecomponents.dapr.io"]
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["get", "patch", "update"]
  resourceNames: ["clusterconnections.conductor.diagrid.io"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings"]
  verbs: ["create"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings"]
  verbs: ["get", "patch", "update"]
  resourceNames: ["dapr-sentry", "dapr-placement", "dapr-operator-admin", "dapr-injector", "dapr-scheduler"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "rolebindings"]
  verbs: ["create"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "rolebindings"]
  verbs: ["get", "patch", "update"]
  resourceNames: ["dapr-sentry", "dapr-placement", "dapr-operator", "dapr-injector", "dapr-scheduler", "secret-reader", "dapr-secret-reader"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "rolebindings"]
  verbs: ["get", "patch", "update"]
  resourceNames: ["dapr-sentry", "dapr-placement", "dapr-operator", "dapr-injector", "dapr-scheduler"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "rolebindings"]
  verbs: ["get", "patch", "update"]
  resourceNames: ["secret-reader", "dapr-secret-reader"]
- apiGroups: [""]
  resources: ["serviceaccounts"]
  verbs: ["get", "patch", "update"]
  resourceNames: ["dapr-operator", "dapr-placement", "dapr-scheduler", "dapr-sentry", "dapr-injector"]
# Diagrid ClusterRole and ClusterRoleBinding
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles"]
  verbs: ["patch", "update"]
  resourceNames:
    - diagrid-agent-dashboard-reader
    - diagrid-agent-dapr-operator-admin
    - diagrid-operator-dashboard-reader
    - diagrid-operator-dapr-operator-admin
  # main role name
    - diagrid
    - diagrid-operator-agent-role
    - diagrid-operator-role
    - dapr-dashboard
    - dashboard-reader
    - dashboard-reader-global
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterrolebindings"]
  verbs: ["get", "patch", "update"]
  resourceNames:
    - diagrid-agent-dapr-operator
    - diagrid-agent-dashboard-reader
    - diagrid-agent-dapr-role-tokenreview-binding
    - diagrid-agent-dashboard-reader-global
    - diagrid-operator-dapr-operator
    - diagrid-operator-dashboard-reader
    - diagrid-operator-dapr-role-tokenreview-binding
    - diagrid-operator-dashboard-reader-global
    - diagrid
    - diagrid-operator-agent-role
    - diagrid-operator-operator-role
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles"]
  verbs: [ "patch", "update"]
  resourceNames:
    - diagrid-operator-role-cm
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["rolebindings"]
  verbs: ["get", "patch", "update"]
  resourceNames:
    - diagrid-operator-operator-role

Dapr workloads​

For Dapr workloads, defining resourceNames is not feasible due to dynamic nature of workload names. Thus, Conductor adheres to the privileges specified in the Dapr Helm chart for the agent to manage Dapr workloads, as outlined in dapr_rbac.

Workload rollouts​

To roll out applications, and ensure they receive the latest version of the Dapr sidecar, it is required to patch the associated Kubernetes Deployment, Daemonset, Statefulset, and Job resources. This is done by modifying an annotation on the workload to trigger a rollout. The annotation used is diagrid.io/restartedAt.

Component initialization checks​

To perform Dapr component initialization checks, the agent needs to have access to read the secrets for some components to be able to attempt the initialization process. Read more about component init checks here.

The list above highlights key resources that require elevated permissions but is not exhaustive. Additional permissions are necessary for the agent to function as intended. Before deployment, review the agent manifest to ensure all required permissions for Conductor are included.

Collected data​

For all Conductor connected clusters, the data sent from the Conductor agent to Diagrid Cloud is as follows:

Cluster data:

  • Dapr Helm chart values
  • Dapr Kubernetes resources (CRDs): Components, Resiliency, Configuration, Subscription
    • Not including sensitive Component information which is obfuscated in the agent
  • Component initialization status

Dapr-enabled app and sidecar data:

  • Container names
  • Container health status
  • Container restart count
  • Pod status and message
  • Pod uptime
  • Count of desired pod replicas and ready pod replicas
  • Dapr annotations

Metrics and logs data:

  • Dapr sidecar (daprd) logs from all Dapr-enabled apps
    • Only error, warning, and fatal log levels are collected
  • Dapr metrics from all Dapr-enabled apps and the Dapr control plane
    • Complete list of Dapr metrics collected found here
  • Resource data for Dapr-enabled app containers, Dapr sidecars, and Dapr control plane containers
    • CPU limit, request, and usage data
    • Memory limit, request, and usage data

Footnotes​

  1. After installing the Kubernetes Metric Server on local Kubernetes clusters you may need to account for this requirement: Kubelet certificate needs to be signed by cluster Certificate Authority (or disable certificate validation by passing --kubelet-insecure-tls to Metrics Server). To pass that flag, use this command on your cluster: kubectl patch deployment metrics-server -n kube-system --type "json" -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--kubelet-insecure-tls"}]' ↩