# Onboard cluster using Kubernetes operator

Dapr Ops Dashboard provides a Kubernetes-native API for declaratively connecting clusters and managing their Dapr control planes. The operator fits naturally into GitOps workflows or any declarative configuration system.

It relies on the `ClusterConnection` Custom Resource Definition (CRD) to connect a Kubernetes cluster to Dapr Ops Dashboard and continuously reconcile the desired configuration, installing, upgrading, or configuring the Dapr control plane as needed.

:::warning Deprecation notice

The Dapr Ops Dashboard Kubernetes operator is undergoing deprecation and will lose support within the near term. Use [Helm-backed installations](https://docs.diagrid.io/dapr-open-source/dapr-ops-dashboard/helm-installation) for GitOps scenarios instead.

:::

## Operator deployment

To deploy the Dapr Ops Dashboard operator, install the [Diagrid CLI](https://docs.diagrid.io/references/conductor/conductor-cli-intro) and run `diagrid operator manifests` to download the Kubernetes manifests.

The operator authenticates with Dapr Ops Dashboard at [https://diagrid.ws/conductor](https://diagrid.ws/conductor) using an API key. Use `--generate-api-key` to create a non-expiring key automatically, or supply/rotate a key with `--api-key <your-api-key>`.

You also need to provide the following arguments depending on your target cluster distribution:

- `--kubernetes-distro`: Kubernetes distribution of target cluster (eks, aks, gke, openshift, native) (default "native")
- `--kubernetes-version`: Kubernetes version of target cluster (default "1.24.2")

<details>
  <summary><strong>Amazon EKS, Kubernetes 1.24.2, auto-generated API key</strong></summary>

  ```bash
  diagrid operator manifests --kubernetes-distro eks --kubernetes-version 1.24.2 --generate-api-key > diagrid-operator.yaml
  ```
</details>

<details>
  <summary><strong>Openshift, Kubernetes 1.23.5, provided API key</strong></summary>

  ```bash
  diagrid operator manifests --kubernetes-distro openshift --kubernetes-version 1.23.5 --api-key <your-api-key> > diagrid-operator.yaml
  ```
</details>

Alternatively, if you want to manage the API Key separately, the operator expects a secret like this:

<details>
  <summary><strong>Operator API key secret manifest</strong></summary>

  ```yaml
  apiVersion: v1
  kind: Secret
  metadata:
    name: diagrid-operator-apikey
    namespace: diagrid-cloud-operator
  type: Opaque
  data:
    apikey: <your-api-key-base64>
  ```
</details>

<details>
  <summary><strong>Local cluster (native), Kubernetes 1.24.2, no API key</strong></summary>

  ```bash
  diagrid operator manifests --kubernetes-distro native --kubernetes-version 1.24.2 > diagrid-operator.yaml
  ```
</details>

For more information about API Keys read [API Keys management](../manage.mdx#api-keys).

## Connect a cluster to Dapr Ops Dashboard

Once the operator is installed, connect the cluster to Dapr Ops Dashboard by creating and applying a `ClusterConnection` CRD.

Copy and tailor the example below to describe your cluster connection and Dapr control plane installation. All values from the upstream Dapr Helm chart are supported and can be copied into `dapr.Configuration.helmValues`.

:::note Note
If you already have Dapr installed with a `HelmResource` or other declarative definition, you will need to delete in order for Dapr Ops Dashboard to manage your Dapr control plane correctly.
:::

<details>
  <summary><strong>Sample ClusterConnection resource</strong></summary>

  ```yaml
  apiVersion: conductor.diagrid.io/v1beta1
  kind: ClusterConnection
  metadata:
    name: default
  spec:
    clusterConfiguration:
      name: my-cluster-connection
      clusterType: prod # Cluster type can be "prod" or "nonprod"
      labels:
        env: demo
      automaticAgentUpgrades: true
      agentConfiguration:
        agentDeployment:
          reporter:
            resources:
              limits:
                memory: "750Mi"
          metricsCollector:
            resources:
              limits:
                memory: "2Gi"
        logsCollectorDaemonSet:
          resources:
            limits:
              memory: "750Mi"
      certificateManagement:
        enabled: true
        renewEveryNumDays: 90
        preferredRenewalTime:
          dayOfWeek: 0 # Sunday
          hour: 23
          minute: 59
    daprConfiguration:
      version: "1.10.2"
      namespace: dapr-system
      helmValues: # Dapr Helm values
        global:
          logAsJson: true
          ha:
            enabled: true
          mtls:
            enabled: false
        dapr_dashboard:
          enabled: false
        dapr_operator:
          runAsNonRoot: true
          watchInterval: 3m
        dapr_placement:
          cluster:
            forceInMemoryLog: true # Required for Openshift clusters
      resourceRollout:
        rolloutAll: true
  ```
</details>

:::danger Important\
A Kubernetes cluster can only be connected to Dapr Ops Dashboard once. The `ClusterConnection` CRD is cluster scoped and must be named default. The value of `spec.clusterConfiguration.name` must be unique per cluster.
:::

## Delete a cluster connection

Before deleting the `ClusterConnection` CRD, remove the Diagrid agent resources. The operator cannot perform this for security reasons, so use the Diagrid CLI to clean up the agent first.

1. Retrieve and copy the ID of the cluster connection you want to delete.

   ```bash
   diagrid clusters list
   ```

2. Use the ID of your cluster connection to retrieve the agent manifests and delete them from your cluster.

   ```bash
   diagrid clusters manifests <cluster-id> | kubectl delete -f -
   ```

3. Delete the `ClusterConnection` CRD from your cluster.

   ```bash
   kubectl delete clusterconnection default
   ```

## Delete the Kubernetes operator

Deleting the `ClusterConnection` CRD does not remove the Dapr Ops Dashboard operator from your cluster. To fully remove Dapr Ops Dashboard management from your cluster, delete the operator manifests from the directory where you created them during [Operator deployment](#operator-deployment).

```bash
kubectl delete -f diagrid-operator.yaml
```

## ClusterConnection CRD specification

The `ClusterConnection` Custom Resource Definition (CRD) is used to connect a Kubernetes cluster to Dapr Ops Dashboard using the Kubernetes operator.

All resource configurations used in the example below are optional and should be adjusted to your needs, but note that the value of `spec.clusterConfiguration.name` must be unique per cluster. Also, be advised that lowering the resource limits may cause the affected components to fail.

### Cluster configuration

| Schema Property                                                                           | Type   | Description                                                                                                                                                                | Required |
| ----------------------------------------------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| `spec .clusterConfiguration`                                                               | object | Specifications for the cluster connection.                                                                                                                                 | Yes      |
| `spec .clusterConfiguration .name`                                                          | string | The name of the cluster as it will appear in Dapr Ops Dashboard. Note this value must be unique per cluster.                                                                        | Yes      |
| `spec .clusterConfiguration .clusterType`                                                   | string | The type of the cluster. It can be either 'nonprod' for non-production clusters or 'prod' for production clusters. The default value is 'nonprod'.                         | No       |
| `spec .clusterConfiguration .labels`                                                        | map    | Labels for the cluster as a map of key-value pairs. Example: `labels: env: demo app: my-app`                                                                               | No       |
| `spec .clusterConfiguration .automaticAgentUpgrades`                                        | bool   | A boolean value that indicates whether the Dapr Ops Dashboard agent upgrades are automatically applied.                                                                             | No       |
| `spec .clusterConfiguration .agentConfiguration`                                            | object | The configuration of the Dapr Ops Dashboard agent.                                                                                                                                  | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment`                            | object | The configuration of the Dapr Ops Dashboard agent Deployment resource.                                                                                                              | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .reporter .replicas`         | int | Number of replicas for the reporter container.                                                                                                                              | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .reporter .resources`         | object | The resource requirements for the reporter container. See [Resources](#resources). Example: `reporter: resources: limits: memory: "750Mi" cpu: "500m"`                     | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .reporter .nodeAffinity`      | object | Kubernetes node affinity for the reporter deployment.                                                                                                                                 | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .metricsCollector .replicas`  | int | Number of replicas for the metrics collector container.                                                                                                                       | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .metricsCollector .resources` | object | The resource requirements for the metrics collector container. See [Resources](#resources). Example: `metricsCollector: resources: limits: memory: "2Gi" cpu: "1"`         | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .metricsCollector .nodeAffinity` | object | Kubernetes node affinity for the metrics collector statefulset.                                                                                                                  | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .metricsCollector .nodeSelector` | object | Kubernetes node selector for the metrics collector statefulset.                                                                                                                  | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .metricsCollector .tolerations` | object | Kubernetes tolerations for the metrics collector statefulset. See [Toleration Specifications](#tolerations)                                                                       | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .nodeSelector`               | map    | The agent deployment node selector specifications.                                                                                                                         | No       |
| `spec .clusterConfiguration .agentConfiguration .agentDeployment .tolerations`                | list   | The agent deployment tolerations specifications. See [Toleration Specifications](#tolerations)                                                                             | No       |
| `spec .clusterConfiguration .agentConfiguration .logsCollectorDaemonSet`                     | object | The configuration options for the logs collector daemonset.                                                                                                                | No       |
| `spec .clusterConfiguration .agentConfiguration .logsCollectorDaemonSet .resources`           | object | The resource requirements for the logs collector daemonset. See [Resources](#resources). Example: `logsCollectorDaemonSet: resources: limits: memory: "750Mi" cpu: "500m"` | No       |
| `spec .clusterConfiguration .agentConfiguration .logsCollectorDaemonSet .nodeAffinity`        | object | Kubernetes node affinity for the logs collector daemonset.                                                                                                                 | No       |
| `spec .clusterConfiguration .agentConfiguration .logsCollectorDaemonSet .nodeSelector`        | map    | The logs collector daemonset node selector specifications.                                                                                                                 | No       |
| `spec .clusterConfiguration .agentConfiguration .logsCollectorDaemonSet .tolerations`         | list   | The logs collector daemonset tolerations specifications. See [Toleration Specifications](#tolerations).                                                                    | No       |
| `spec .clusterConfiguration .agentConfiguration .proxyConfiguration .httpProxy`               | string | The HTTP proxy configuration for the agent deployment and daemonset.                                                                                                       | No       |
| `spec .clusterConfiguration .agentConfiguration .proxyConfiguration .httpsProxy`              | string | The HTTPS proxy configuration for the agent deployment and daemonset.                                                                                                      | No       |
| `spec .clusterConfiguration .agentConfiguration .proxyConfiguration .noProxy`                 | string | The no proxy configuration for the agent deployment and daemonset.                                                                                                         | No       |
| `spec .clusterConfiguration .agentConfiguration .additionalCACerts`                          | string | The additional CA certificates to trust for the agent deployment and daemonset.                                                                                            | No       |
| `spec .clusterConfiguration .certificateManagement`                                         | object | The configuration options for Dapr mTLS automatic certificate renewal.                                                                                                     | No       |
| `spec .clusterConfiguration .certificateManagement .enabled`                                 | bool   | A boolean value that indicates whether Dapr mTLS automatic certificate renewal is enabled. Default: `false`.                                                               | No       |
| `spec .clusterConfiguration .certificateManagement .renewEveryNumDays`                       | int    | The number of days after which the Dapr mTLS certificate will be renewed. Default: 30. Options: 30, 60, 90, 365.                                                           | No       |
| `spec .clusterConfiguration .certificateManagement .preferredRenewalTime`                    | object | The Dapr mTLS automatic certificate renewal preferred time.                                                                                                                | No       |
| `spec .clusterConfiguration .certificateManagement .preferredRenewalTime .dayOfWeek`          | int    | The preferred day of the week for Dapr mTLS automatic certificate renewal. Options: 0 (Sunday) - 6 (Saturday).                                                             | No       |
| `spec .clusterConfiguration .certificateManagement .preferredRenewalTime .hour`               | int    | The preferred hour for Dapr mTLS automatic certificate renewal. Options: 0 - 23.                                                                                           | No       |
| `spec .clusterConfiguration .certificateManagement .preferredRenewalTime .minute`             | int    | The preferred minute for Dapr mTLS automatic certificate renewal. Options: 0 - 59.                                                                                         | No       |

### Dapr configuration

| Schema Property                                     | Type    | Description                                                                                                                                                   | Required |
| --------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| `spec.daprConfiguration`                            | string  | The Dapr configuration options.                                                                                                                               | Yes      |
| `spec.daprConfiguration.version`                    | string  | The Dapr control plane version specified by semantic versioning. For example:`"1.12.2"`                                                                       | Yes      |
| `spec.daprConfiguration .namespace`                  | string  | The namespace for that the Dapr control plane will be deployed into.                                                                                          | Yes      |
| `spec.daprConfiguration .helmValues`                 | yaml    | The Dapr Helm values in YAML format. See the Helm chart [configuration options](https://github.com/dapr/dapr/blob/master/charts/dapr/README.md#configuration) | Yes      |
| `spec.daprConfiguration .resourceRollout`            | object  | The resource rollout options for Dapr-enabled applications.                                                                                                   | Yes      |
| `spec.daprConfiguration .resourceRollout.rolloutAll` | boolean | A boolean value that indicates whether all Dapr-enabled applications should be rolled out.                                                                    | Yes      |

### Tolerations

Standard Kubernetes taints and tolerations specifications. Read more at [Kubernetes Taints and Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/).

| Schema Property                    | Type   | Description                               | Required |
| ---------------------------------- | ------ | ----------------------------------------- | -------- |
| `.tolerations`                     | list   | An array of tolerations specifications.   | No       |
| `.tolerations[].key`               | string | The key of the toleration.                | Yes      |
| `.tolerations[].operator`          | string | The operator of the toleration.           | Yes      |
| `.tolerations[].value`             | string | The value of the toleration.              | Yes      |
| `.tolerations[].effect`            | string | The effect of the toleration.             | Yes      |
| `.tolerations[].tolerationSeconds` | int64  | The toleration seconds of the toleration. | No       |

### Resources

Standard Kubernetes resource requests and limits of pods and containers. Read more at [Resource requests and limits of Pod and container](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#resource-requests-and-limits-of-pod-and-container).

| Schema Property            | Type   | Description                        | Required |
| -------------------------- | ------ | ---------------------------------- | -------- |
| `.resources`               | object | The resource specification object. | No       |
| `.resources.limits`        | object | The resource limits object.        | No       |
| `.resources.limits.memory` | string | The memory limit.                  | No       |
| `.resources.limits.cpu`    | string | The CPU limit.                     | No       |
