Using Catalyst APIs with Azure Resources

Deploy two generic applications to Azure to interact with the Pub/Sub, Request/Reply and Key/Value APIs. In this tutorial, you will:

1. Deploy and connect to Azure infrastructure services to configure the Catalyst APIs: Azure Storage and Azure Service Bus
2. Deploy two sample applications to an Azure container hosting platform: An inbound application to test calling Catalyst APIs, and an outbound application to receive the resulting calls/events.
3. Create and configure Diagrid App IDs to connect to and from the Azure applications
4. Invoke the Catalyst Pub/Sub, Request/Reply and Key/Value APIs

The source code used for this tutorial can be found in the catalyst-samples GitHub repository.

Prerequisites

Requirement	Instructions
Diagrid Account	Catalyst is currently an early-access product. If you have not already been invited to create an account, join the waitlist to request access.
Diagrid CLI	Install the Diagrid CLI
GitHub Account	Get one for free.
Azure Account	If you don't have one, create an account for free. You need the Contributor or Owner permission on the Azure subscription to proceed.
Azure CLI	Install the Azure CLI.
Jq	Install the jq cli.

Log in to Azure

Sign in to Azure.

az login

Ensure you're running the latest version of the Azure CLI via the upgrade command.

az upgrade

Set environment variables

Set environment variables which will be used in the deployment of Azure resources:

export RESOURCE_GROUP="catalyst-rg"
export LOCATION="eastus"
export SERVICEBUS_NAMESPACE="catalyst-ns"
export STORAGEACCOUNT_NAME="orderkvstore"

Deploy infrastructure services to Azure

This tutorial will require deploying the following Azure infrastructure resources:

• Azure Service Bus for pub/sub: A Service Bus Namespace and Topic will be created for enabling the publisher/subscriber pattern.
• Azure Storage for key/value state management: A DynamoDB table will be created for persisting key/value pairs.

First, create an Azure Resource group to host the resources created as part of the tutorial:

az group create --name $RESOURCE_GROUP --location $LOCATION

Deploy Azure Service Bus

Create an Azure Service Bus Namespace.

az servicebus namespace create --resource-group $RESOURCE_GROUP --name $SERVICEBUS_NAMESPACE --location $LOCATION  --sku Standard

Create a topic in the Service Bus Namespace which will provide a one-to-many form of communication via the publish and subscribe pattern.

az servicebus topic create --name orders --namespace-name $SERVICEBUS_NAMESPACE --resource-group $RESOURCE_GROUP

Retrieve the Service Bus connection string and store in an environment variable. This credential will be used by the Diagrid Pub/Sub API to connect to the topic created above.

export SB_CONNECTION_STRING=`az servicebus namespace authorization-rule keys list --resource-group $RESOURCE_GROUP --namespace-name $SERVICEBUS_NAMESPACE --name RootManageSharedAccessKey --query primaryConnectionString -o tsv`

Recommended: Confirm successful retrieval by running echo $SB_CONNECTION_STRING

Deploy Azure Storage KV Store

Create an Azure Storage Account which will be used by the inbound app to persist key/value pairs.

az storage account create --name $STORAGEACCOUNT_NAME --resource-group $RESOURCE_GROUP --location $LOCATION --sku Standard_RAGRS --kind StorageV2

Retrieve the Storage Account key and store in an environment variable. This credential will be used by the Key/Value API to connect to the storage account created above.

export STORAGE_KEY=`az storage account keys list -g $RESOURCE_GROUP -n $STORAGEACCOUNT_NAME | jq -r '.[0].value'`

Recommended: Confirm successful retrieval by running echo $STORAGE_KEY

Create Diagrid Catalyst resources

Log in

Authenticate to your Diagrid Catalyst organization using the following command:

diagrid login 

Confirm your login was successful:

diagrid whoami

Create project

If you do not have an existing project available within your organization, create a new Catalyst project.

diagrid project create catalyst-project

To ensure this project is set as the default project in the Diagrid CLI, run:

diagrid project use catalyst-project

Create App IDs

In Catalyst, each application is represented via a corresponding remote identity, known as an App ID. An App ID functions as the single point of contact for all interactions between a specific application and the Catalyst APIs.

Create two App IDs: One for the sample app which will make inbound calls to Catalyst, and the other for the sample app which will receive outbound calls from Catalyst.

diagrid appid create inbound 
diagrid appid create outbound

Retrieve the API token for the inbound App ID. App ID API tokens are used to authenticate API requests to Catalyst from external callers.

export INBOUND_API_TOKEN=$(diagrid appid get inbound -o json  | jq -r '.status.apiToken')

Recommended: Confirm successful retrieval by running echo $INBOUND_API_TOKEN

Create Azure infrastructure connections

In order to use the infrastructure resources deployed above with the Catalyst APIs, two connections are needed:

• az-pubsub: Pub/Sub Connection for interacting with Azure Service Bus using the Pub/Sub API. Connection access will be scoped to both the inbound publisher and outbound consumer App IDs.
• az-kvstore: KV Connection for the Azure Storage Account. Connection will be scoped to the inbound App ID.

Create the Azure Service Bus Pub/Sub Connection:

 diagrid connection create az-pubsub --type pubsub.azure.servicebus.topics --metadata connectionString=$SB_CONNECTION_STRING --scopes inbound,outbound 

Create the Azure Storage KV Connection:

 diagrid connection create az-kvstore --type state.azure.blobstorage --metadata accountName=$STORAGEACCOUNT_NAME --metadata accountKey=$STORAGE_KEY --metadata containerName="orders" --scopes inbound 

Recommendation: To ensure the connections were successfully deployed run diagrid connection list

Create topic subscription

To configure the outbound app to receive messages published by the inbound app using the Pub/Sub API, create a topic subscription targeting the az-pubsub connection with the following details:

• Subscription name: az-subscription
• Target Pub/Sub connection: az-pubsub
• Topic on which to listen for incoming messages: orders
• Default route on the outbound app for message delivery: /pubsub/neworders
• Subscribing App IDs (scopes): outbound

diagrid subscription create az-subscription --connection az-pubsub --topic orders --route /pubsub/neworders --scopes outbound

Recommendation: To ensure the subscription was successfully created run diagrid subscription list

Deploy containerized applications to Azure

To test Catalyst APIs which require inbound calls to Catalyst and outbound calls to an application i.e. Pub/Sub and Request/Reply, you will deploy two sample applications:

• An inbound app for making calls to the Catalyst APIs
• An outbound app for receiving calls from the Catalyst APIs

To make API calls from the application code to Catalyst, the inbound application uses the OSS Dapr SDKs. Environment variables will be configured to enable the Dapr Client to connect to Catalyst instead of a locally-running Dapr sidecar.

Learn more about using the Dapr SDKs to access Catalyst APIs here.

The applications have already been containerized and pushed to a public registry. The sample docker images are available for Python, JavaScript, .NET and Java. You can choose to use any combination of languages for the inbound and outbound apps.

Deploy inbound application

Run the below command to set the coding language you want to deploy for the inbound application:

.NET

export INBOUND_LANGUAGE=csharp

Python

export INBOUND_LANGUAGE=python

JavaScript

export INBOUND_LANGUAGE=javascript

Java

export INBOUND_LANGUAGE=java

Retrieve the HTTP and gRPC endpoints for your Catalyst project. These endpoints will be used by the inbound application to make calls to the Catalyst APIs.

export DAPR_HTTP_ENDPOINT=`diagrid project get -o json | jq -r '.status.endpoints.http.url'`
export DAPR_GRPC_ENDPOINT=`diagrid project get -o json | jq -r '.status.endpoints.grpc.url'`

Recommended: Confirm successful retrieval by running echo $DAPR_HTTP_ENDPOINT && echo $DAPR_GRPC_ENDPOINT

Select the platform where you want to deploy the inbound (caller) application.

Container Apps

Ensure you have the necessary extensions installed and providers registered for working with Azure Container Apps (ACA).

az extension add --name containerapp --upgrade
az provider register --namespace Microsoft.App
az provider register --namespace Microsoft.OperationalInsights

Deploy the inbound application to ACA. This command will:

• Deploy the sample application in the language of your choice to an ACA environment
• Configure the environment variables required for the application to make Catalyst API calls
• Expose the container app on a publically-accessible endpoint

az containerapp up \
--name inbound \
--resource-group $RESOURCE_GROUP \
--ingress external --target-port 5001 \
--image 'public.ecr.aws/diagrid/'${INBOUND_LANGUAGE}'-sample:latest' \
--env-vars DAPR_HTTP_ENDPOINT="$DAPR_HTTP_ENDPOINT" DAPR_GRPC_ENDPOINT="$DAPR_GRPC_ENDPOINT" DAPR_API_TOKEN="$INBOUND_API_TOKEN" PUBSUB_NAME="az-pubsub" KVSTORE_NAME="az-kvstore" INVOKE_APPID="outbound"

Retrieve the application FQDN for the inbound application.

export INBOUND_APP_ENDPOINT=https://`az containerapp show --name inbound --resource-group $RESOURCE_GROUP | jq -r .properties.configuration.ingress.fqdn`

Recommended: Run echo $INBOUND_APP_ENDPOINT to ensure the FQDN was successfully retrieved.

Container Instances

Deploy the inbound application to Azure Container Instances (ACI). This command will:

• Deploy the sample application in the language of your choice as a container instance
• Configure the environment variables required by the application to make Catalyst API calls
• Expose the container instance on a publically-accessible endpoint

info

The DNS name label parameter value must be unique so you may need to add a unique prefix to avoid a conflict.

az container create --name inbound \
--resource-group $RESOURCE_GROUP \
--ip-address public --dns-name-label inbound --ports 5001 \
--image 'public.ecr.aws/diagrid/'${INBOUND_LANGUAGE}'-sample:latest' \
--environment-variables DAPR_HTTP_ENDPOINT="$DAPR_HTTP_ENDPOINT" DAPR_GRPC_ENDPOINT="$DAPR_GRPC_ENDPOINT" DAPR_API_TOKEN="$INBOUND_API_TOKEN" PUBSUB_NAME="az-pubsub" KVSTORE_NAME="az-kvstore" INVOKE_APPID="outbound"

Retrieve the application FQDN for the inbound application.

export INBOUND_APP_ENDPOINT=http://`az container show --name inbound --resource-group $RESOURCE_GROUP | jq -r .ipAddress.fqdn`:5001

Recommended: Run echo $INBOUND_APP_ENDPOINT to ensure the FQDN was successfully retrieved.

App Service

Set the following environment variables for deploying to Azure App Service.

export APP_SUFFIX="your-unique-suffix"

Create an App Service Plan to host the inbound application.

az appservice plan create --name catalyst-apps --resource-group $RESOURCE_GROUP --sku free --is-linux

Deploy a web app for the inbound application.

az webapp create --name inbound-$APP_SUFFIX --resource-group $RESOURCE_GROUP --plan catalyst-apps \
-i 'public.ecr.aws/diagrid/'${INBOUND_LANGUAGE}'-sample:latest'

Set the appropriate app settings required by the application to make Catalyst API calls.

az webapp config appsettings set -g $RESOURCE_GROUP -n inbound-$APP_SUFFIX \
--settings PUBSUB_NAME="az-pubsub" KVSTORE_NAME="az-kvstore" DAPR_HTTP_ENDPOINT="$DAPR_HTTP_ENDPOINT" DAPR_GRPC_ENDPOINT="$DAPR_GRPC_ENDPOINT" DAPR_API_TOKEN="$INBOUND_API_TOKEN" WEBSITES_PORT=5001 INVOKE_APPID="outbound"

Restart the web app to ensure the newly added app settings are available to the application.

az webapp restart --resource-group $RESOURCE_GROUP --name inbound-$APP_SUFFIX

tip

It may take up to 5 minutes for the App Service deployment to successfully complete depending on language. For example, Java web app deployment may take more time.

Retrieve the application FQDN for the inbound application.

export INBOUND_APP_ENDPOINT=https://`az webapp show --name inbound-$APP_SUFFIX --resource-group $RESOURCE_GROUP | jq -r .defaultHostName`

Recommended: Run echo $INBOUND_APP_ENDPOINT to ensure the FQDN was successfully retrieved.

Deploy outbound application

Run the below command to set the coding language you want to deploy for the outbound application:

.NET

export OUTBOUND_LANGUAGE=csharp

Python

export OUTBOUND_LANGUAGE=python

JavaScript

export OUTBOUND_LANGUAGE=javascript

Java

export OUTBOUND_LANGUAGE=java

Select the platform where you want to deploy the outbound (receiver) application.

Container Apps

Ensure you have the necessary extensions installed and providers registered for working with Azure Container Apps.

az extension add --name containerapp --upgrade
az provider register --namespace Microsoft.App
az provider register --namespace Microsoft.OperationalInsights

Deploy the outbound application to ACA. This command will:

• Deploy the sample application in the language of your choice to an ACA environment
• Expose the container app on a publically-accessible endpoint

az containerapp up \
--name outbound \
--resource-group $RESOURCE_GROUP \
--ingress external --target-port 5001 \
--image 'public.ecr.aws/diagrid/'${OUTBOUND_LANGUAGE}'-sample:latest'

Retrieve the application FQDN for the outbound application.

export OUTBOUND_APP_ENDPOINT=https://`az containerapp show --name outbound --resource-group $RESOURCE_GROUP | jq -r .properties.configuration.ingress.fqdn`

Recommended: Run echo $OUTBOUND_APP_ENDPOINT to ensure the FQDN was successfully retrieved.

Container Instances

Deploy the outbound application to ACI. This command will:

• Deploy the sample application in the language of your choice as a container instance
• Expose the container instance on a publically-accessible endpoint

info

The DNS name label parameter value must be unique so you may need to add a unique prefix to avoid a conflict.

az container create --name outbound \
--resource-group $RESOURCE_GROUP \
--ip-address public --dns-name-label outbound --ports 5001 \
--image 'public.ecr.aws/diagrid/'${OUTBOUND_LANGUAGE}'-sample:latest'

Retrieve the application FQDN for the outbound application.

export OUTBOUND_APP_ENDPOINT=http://`az container show --name outbound --resource-group $RESOURCE_GROUP | jq -r .ipAddress.fqdn`:5001

Recommended: Run echo $OUTBOUND_APP_ENDPOINT to ensure the FQDN was successfully retrieved.

App Service

If you did not use App Service for the inbound application, complete these steps, otherwise proceed to the deployment command.

export APP_SUFFIX="your-unique-suffix"

Create an App Service Plan to host the outbound application or use the existing plan if previously created.

az appservice plan create --name catalyst-apps --resource-group $RESOURCE_GROUP --sku free --is-linux

Deploy a web app for the outbound application.

az webapp create --name outbound-$APP_SUFFIX --resource-group $RESOURCE_GROUP --plan catalyst-apps \
-i 'public.ecr.aws/diagrid/'${OUTBOUND_LANGUAGE}'-sample:latest'

tip

It may take up to 5 minutes for the App Service deployment to successfully complete depending on language. For example, Java web app deployment may take more time.

Retrieve the application FQDN for the outbound application.

export OUTBOUND_APP_ENDPOINT=https://`az webapp show --name outbound-$APP_SUFFIX --resource-group $RESOURCE_GROUP | jq -r .defaultHostName`

Recommended: Run echo $OUTBOUND_APP_ENDPOINT to ensure the FQDN was successfully retrieved.

Update the outbound App ID with an app endpoint

To ensure incoming App ID requests and events can be forwarded to the outbound application, update the outbound App ID with the outbound application endpoint and wait for the operation to complete.

diagrid appid update outbound --app-endpoint $OUTBOUND_APP_ENDPOINT -w

Interact with the Catalyst APIs

You've now successfully deployed the application code and all of the necessary Catalyst resources to begin invoking the APIs.

Use the below commands to kick off API calls from the inbound application to the Catalyst APIs.

Publish/Subscribe

curl -i -X POST $INBOUND_APP_ENDPOINT/pubsub/orders  -H "Content-Type: application/json" -d '{"orderId":1}'

Request/Reply

curl -i -X POST $INBOUND_APP_ENDPOINT/invoke/orders  -H "Content-Type: application/json" -d '{"orderId":1}'

Key/Value

Save a key/value pair

curl -i -X POST $INBOUND_APP_ENDPOINT/kv/orders  -H "Content-Type: application/json" -d '{"orderId":1}'

Update the value of orderId in the data payload with a new key to store another key/value pair.

Retrieve an existing key/value pair

curl -i -X GET $INBOUND_APP_ENDPOINT/kv/orders/1

Delete an existing key/value pair

curl -i -X DELETE $INBOUND_APP_ENDPOINT/kv/orders/1

Validate APIs via logs

To validate the APIs are woring as expected, use the commands below to inspect your application logs in a secondary terminal as you make requests.

Inbound app logs

Container Apps

az containerapp logs show --name outbound -g $RESOURCE_GROUP --type console --follow --tail 30 

Container Instances

az container logs --resource-group $RESOURCE_GROUP --name inbound --follow 

App Service

az webapp log tail --name inbound-$APP_SUFFIX --resource-group $RESOURCE_GROUP --provider http

Outbound app logs

Container Apps

az containerapp logs show --name outbound --resource-group $RESOURCE_GROUP --type console --follow --tail 30

Container Instances

az container logs --resource-group $RESOURCE_GROUP --name outbound --follow

App Service

az webapp log tail --name outbound-$APP_SUFFIX --resource-group $RESOURCE_GROUP --provider http

App ID logs

To view the App ID logs for both the inbound and outbound applications run the following command

diagrid appid logs --all --tail 5 --follow

Summary

In this tutorial, you were able to instrument two applications with the Catalyst APIs to achieve streamlined communication across application hosting platforms and easy onboarding of supporting infrastructure services, all while simplifying the application code and achieving a greater level of development consistency.

Clean up resources

Once you're done testing the APIs, you can use these commands to clean up the Azure and Catalyst resources you created in this tutorial.

Delete resource group

az group delete --name $RESOURCE_GROUP

Delete Catalyst resources

diagrid appid delete inbound 
diagrid appid delete outbound
diagrid connection delete az-pubsub
diagrid connection delete az-kvstore 

If you want to delete the entire Catalyst project, including the managed infrastructure resources, run the diagrid project delete command.

diagrid project delete catalyst-project