Skip to main content

Diagrid Pub/Sub

Diagrid offers a fully managed Pub/Sub service, designed to facilitate efficient, scalable, and reliable message-driven communication between cloud-native applications.

Behaviors​

Diagrid’s Pub/Sub service integrates seamlessly with cloud-native applications, supporting at-least-once message delivery guarantees, dead letter topics, and both streaming and declarative subscription models. With automatic topic creation, developers can focus on building functionality without manual setup.

A producer sends messages to an input channel targeting a topic, without needing to know the receiving App ID. Similarly, consumers subscribe to topics and receive messages from an output channel, without knowledge of the producer's identity.

API​

The Diagrid Pub/Sub service integrates with the Publish API, even supporting the publish and subscribe to bulk messages. This API provides a simple interface for efficient message exchange between App IDs within a project.

Access Control​

By default, any App ID within the same project can publish to and consume messages from the Diagrid Pub/Sub service. To limit access, you can configure the Pub/Sub Component scopes to restrict usage to specific App IDs, enhancing security and control.

While applications in a project share the underlying infrastructure, every project is assigned a unique identifier that ensures distinct topic names across projects. Additionally, the lifecycle of the Diagrid Pub/Sub component impacts broker data. When the component is deleted, all associated App ID topics and messages within the project are automatically cleaned up.

All data is encrypted at rest, and abiding by secure communication using the Dapr API and App token authentication.

Resiliency​

To ensure fault tolerance and high availability, all outbound and inbound requests to the Diagrid Pub/Sub infrastructure follow resiliency policies. In case of failure, the system retries operations up to 3 times with a 5-second constant delay between attempts, mitigating transient issues such as network disruptions. Outbound requests have a 10-second timeout, while inbound message processing must complete within 15 seconds to prevent failures.

Observability​

The Catalyst UI includes a Data Explorer tool, providing valuable insights into topics across all App IDs, including subscriber counts and total messages This facilitates easy monitoring, debugging, and optimization of the Diagrid Pub/Sub messaging system.

Outbox​

The Diagrid Pub/Sub service supports the outbox pattern, enabling a single transactional operation across both the Diagrid KV Store and Pub/Sub. This feature is automatically enabled for projects using Diagrid Services. In this model, keys are temporarily stored and then deleted from the Diagrid KV Store, and an outbox topic is generated for each App ID within the project.

Quotas​

Messages are not removed when consumed. When a topic is full, then old messages are deleted as we have infinite retention time.

Additionally, we enforce limits on:

  • Max topic count: 10
  • Max Topic data retention: 50MB
  • Max Topic segment data: 50MB
  • Session timeout: 30s

For more details on the quotas and limits regarding the Diagrid Pub/Sub, visit the Support documentation.