Clusters

This document details the various insights in Conductor which provide visibility into the overall health of onboarded Dapr Kubernetes clusters. For information on managing cluster connections in Conductor, please see the cluster configuration overview.

To ensure operators can easily assess the state of a Dapr cluster, Conductor provides an all-in-one dashboard which displays critical information pertaining to the Dapr control plane and data plane on a per-cluster basis. Conductor also provides additional cluster insights which will be detailed in this document.

Cluster insights

The cluster insights view provides easy visibility into the following:

1. Cluster control plane health and uptime
2. Conductor agent status logs
3. Critical Dapr control plane metrics issues
4. Proactive recommendations for optimizing Dapr operations within the cluster via advisor

Dapr control plane health

Ideally the Dapr control plane in the cluster is always healthy, however this may not be the case. To explore the data driving the Control Plane health status, select the Healthy or Unhealthy indicator under control plane health.

The control plane health view will show each Dapr control plane pod along with data which can be used to hone in on potential issues such as pod restart count and status message.

In addition to pod details, control plane metrics are also captured to provide further insights into potential metrics issues.

Note

Critical metrics issues for the control plane are covered by default notification rules, meaning any critical metrics error visible in the metrics graph will likely trigger a notification rule to fire.

Conductor agent health

The Condutor agent will appear as either Online or Offline. To view status logs to determine the most recent actions performed by the agent, select the status indicator under agent health.

Dapr application and component insights

In addition to the cluster control plane insights and recommendations, Conductor aggregates data captured from the Dapr data plane and Dapr-enabled workloads in the cluster to provide a more holistic view of overall cluster health. These insights include:

1. Fatal logs issues, critical metrics issues, and high impact advisiories across all cluster applications
2. Fatal logs issues, initialization errors, and high impact advisiories across all cluster components

Apps Graph

The dashboard also includes an embedded view of the the Apps Graph. The Apps Graph displays applications as individual nodes which may be interconnected via Dapr service invocation and pub/sub API calls. Individual application nodes and components can be isolated on to provide additional context and data including potential metrics issues. For more details see application visualization.

Cluster inventory

Within the context of a single cluster, visibility is provided across all Dapr resources including:

• Dapr-enabled applications
• Components
• Configurations
• Subscriptions
• Resiliency Policies
• Actors

Applications

When inspecting a cluster, you can take a deeper look at the applications deployed across all namespaces. If the an application insights status appears as Issues Detected, the cluster dashboard will likely have either critical logs issues, critical metrics issues, or high advisories which have not be addressed in the App Insights section.

In addition to viewing insights across all applications, you can also view data around current CPU and memory utilization of the pod, as well as the Dapr version of the injected sidecar. At a glace, the resource usage data can provide an easy way to identify the most resource intensive applications in the cluster.

Reminder

Upgrading the Dapr control plane version does not automatically upgrade all Dapr sidecars to ensure they are using the newest version. To rollout applications after an upgrade has taken place, you can choose to rollout an individual application, select a subset of applications to rollout, or you can rollout all applications via the top-level action menu on the applications tab in the console.

To dive into application insights in more detail, visit applications

Components

Dapr components connect applications to underlying infrastructure via the Dapr APIs. Conductor provides visibility into all Dapr components deployed on a Dapr cluster.

For every component, core details such as component name, type, namespace and scoped applications are displayed.

In addition, the component is assigned an Initialization Status which provides an easy way for users to catch potential component configuration errors. The below section covers this concept in more detail.

Component Initialization

Dapr component initialization runs on a minute-by-minute basis to ensure all components running in a cluster are configured correctly before being loaded by Dapr-enabled applications. The component initialization is comprised of the following steps:

1. The Conductor agent retrieves a list of all Dapr resources of kind Component across namespaces in the cluster
2. A programmatically instantiated version of each component will be created inside the agent, where its metadata fields will be applied and the associated Init function will be executed. This result will then be reported back to Diagrid Cloud.
3. The result is shown in the Conductor console in the Initialization Status column of both the Cluster and App Components lists. A green checkmark is displayed when the component has been initialized successfully, and a red icon for an error, along with the error message and timestamp from the last time the error occurred. This result can also be seen in each component's manifest in the initStatus field by clicking on "View Raw". The absence of the initStatus field indicates that the component was initialized successfully.

Additional resource insights

Configurations

All Dapr configuration resources deployed to a cluster can be listed and viewed by navigating to the Configurations tab in the console. Configuration can be assigned to a Dapr-enabled pod via the dapr.io/config annotation. For insight into the various sidecar configuration settings supported by Dapr, see sidecar configuration settings.

Subscriptions

All Dapr subscription resources deployed to a cluster can be listed and viewed by navigating to the Subscriptions tab in the console. The cluster-level subscriptions view provides information about the types, topics, associated pub/sub components, and routing information for each subscription.

Resiliency Policies

All Dapr resiliency resources deployed to a cluster can be listed and viewed by navigating to the Recilieny Policies tab in the console.

Actors

Dapr implements the virtual actor pattern, providing an approach to manage concurrency, fault tolerance and resource management of distributed apps. To make monitoring actors in a cluster easier for operators, Conductor provides:

• A consolidated view of the various Dapr actor types present in the cluster along with the active actor count
• Actor invocation metrics graphs to depict the frequency of direct actor calls, as indicated by the Count of Completed Requests metric (dapr_http_client_completed_count). This metric originates from the Dapr HTTP client which is used by the Dapr SDK to facilitate actor communication.
• Timer metrics graphs to depict the frequency at which an actor's timer is fired. It corresponds to the Number of Actor Timers Fired Requests (dapr_runtime_actor_timers_fired_total) metric.
• Reminders metrics graphs to depict the frequency at which an actor's reminder is fired. It corresponds to the Number of Actor Reminders Fired Requests (dapr_runtime_actor_reminders_fired_total) metric.

Info

The data points on the actor metrics graphs reflect the average per-second rate over the past five minutes.