The LEGO version of the is from this Rebrickable Eurasian Blue Tit - Cyanistes caeruleus MOC, the other one is real.

Created at my kitchen table and the photo is shot in my backyard.

The Microsoft Agent Framework support creating agents thats uses the generic OpenAI Chat Completion services. This means that you can use the Microsoft Agent Framework to create agents that uses Amazon Bedrock models by using the OpenAI compatible API provided by Amazon Bedrock.

Creating a Bedrock Agent with Microsoft Agent Framework

To create a Bedrock Agent with the Microsoft Agent Framework, you need to configure the OpenAIClient to use the Bedrock endpoint and provide the necessary Api Key. Below is an example of how to set up a Bedrock Agent using C#:

#:package Microsoft.Agents.AI.OpenAI@1.0.0-preview.251110.2

using System.ClientModel;
using OpenAI;
using OpenAI.Chat;

OpenAIClient client = new(
    new ApiKeyCredential("<your_aws_bedrock_api_key>"),
    new OpenAIClientOptions()
    {
        Endpoint = new Uri("https://bedrock-runtime.<your_aws_region>.amazonaws.com/openai/v1"),
    });

var chatCompletionClient = client.GetChatClient("openai.gpt-oss-120b-1:0");

ChatCompletion chatCompletion = await chatCompletionClient.CompleteChatAsync("Hello, how can I use AWS Bedrock with Microsoft Agent Framework?");

Console.WriteLine(chatCompletion.Content[0].Text.Trim());

Happy building! 🚀

View the Album

This integration start with a third party putting a message on our queue. Because the amount of message we think this integration would process we would like to make this integration serverless. This means no service running 24-7 that is constantly polling for work to do, but a process that is triggered when a new message has arrived.

Integration overview

In this post I will show you how to use AWS SQS and EventBridge Pipes to send messages to ECS. The integration consists of the following components:

• SQS Queue: The queue where the third party put the messages. Nothing special about this queue, just a standard SQS queue.
• EventBridge Pipe: The pipe that connects the SQS queue to the ECS task.
• ECS Task: The task that processes the messages from the SQS queue. This task is running a custom .NET application that communicates with the backend service using WCF.

EventBridge Pipe

Creating an EventBridge Pipe is a straightforward process. The pipe will listen to the SQS queue and trigger the ECS task when a new message arrives. When creating the pipe, you need to specify the source (SQS queue) and the target (ECS task). You can also configure the pipe to filter messages or enrich messages based on certain criteria, but for this example, we will keep it simple and process all messages as they arrive.

CloudFormation Template

AWSTemplateFormatVersion: '2010-09-09'
Description: CloudFormation template for EventBridge PipeSQS-Pipe-ECS-Integration
Resources:
  MySQSQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: MyQueue

  MyECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: MyECSCluster

  MyECSTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: MyTaskDefinition
      ContainerDefinitions:
        - Name: MyContainer
          Image: my-docker-image
          Memory: 512
          Cpu: 256
          Essential: true

  MyEventBridgePipe:
    Type: AWS::Events::Pipe
    Properties:
      Name: MyQueueToECSPipe
      DesiredState: RUNNING
      Source: !GetAtt MySQSQueue.Arn
      Target: !GetAtt MyECSCluster.Arn
      TargetParameters:
        EcsTaskParameters:
          TaskDefinitionArn: MyECSTaskDefinition.Arn
          TaskCount: 1
          NetworkConfiguration:
            awsvpcConfiguration: {}
          EnableECSManagedTags: true
          EnableExecuteCommand: false

This (partial complete) CloudFormation template creates the necessary resources for the integration. It creates an SQS queue, an ECS cluster, an ECS task definition, and an EventBridge Pipe that connects the SQS queue to the ECS task.

This Pipe will automatically trigger the ECS task when a new message arrives in the SQS queue. The ECS task will then start up. It doesn’t receive anything from the SQS queue, it will just start up and run the code in the container.

If the Pipe is configured correctly, it will start the ECS task when a new message arrives in the SQS queue. The message is not passed to the ECS task, but the task will start up and run the code in the container. When the container is started, the message is already removed from the SQS queue. This means that the ECS task will not receive the message automatically.

You can pass the message to the ECS task by using $.body in the template. This will pass the message body to the ECS task as an environment variable. You can then read this environment variable in your code and process the message accordingly.

  MyEventBridgePipe:
    Type: AWS::Events::Pipe
    Properties:
      TargetParameters:
        EcsTaskParameters:
          ... # other parameters
          Overrides:
            ContainerOverrides:
              - Name: MyContainer
                Environment:
                  - Name: MESSAGE_BODY
                    Value: "$.body"

It is also possible to pass the message body as a startup argument to the ECS task. In the ContainerOverrides section of the EcsTaskParameters, you can specify the command to run when the container starts. This allows you to pass the message body as an argument to your application.

  MyEventBridgePipe:
    Type: AWS::Events::Pipe
    Properties:
      TargetParameters:
        EcsTaskParameters:
          ... # other parameters
          Overrides:
            ContainerOverrides:
              - Name: !Ref ContainerName
                Command: 
                  - "dotnet"
                  - "/app/Processor.Host.dll"
                  - "--body"
                  - "$.body"

No resilience integration

When the Pipe can not start the ECS task because, for example: incorrect configuration, unknown task definition, or not the right permission to do so, it will not delete the message from the SQS queue. This means that if the ECS task fails to start, the message will remain in the SQS queue and can be processed again later. However, if the Pipe successfully starts the ECS task, it will delete the message from the SQS queue.

Warning: When the Pipe has successfully started the ECS task, it will not wait for the task to complete but it will delete the message from the SQS queue. This means that if the ECS task fails to process the message, the message will be lost. This will make this integration not resilient, a failure in the ECS task will not result in the message being retried.

Conclusion

This is not a resilient integration, but a simple way to start a process based on a message that is send to an SQS queue using EventBridge Pipes and ECS. The integration is serverless, meaning that there is no need to run a service 24-7 that is constantly polling for work to do. Instead, the ECS task is triggered when a new message arrives in the SQS queue, making it a cost-effective solution for processing messages.

View the Album

When changing the Bicep files in a pull request, you can use the az bicep lint --file $file command to validate the Bicep files. This command will validate the syntax of the Bicep files and will also validate if the resources are valid. This is a great way to validate your Bicep files before deploying them to Azure.

When adding or removing a parameter from the Bicep template, I often forget to change all the Bicepparam files and when deploying to the Production environment in Azure, the deployment wiil fail because the Bicepparam file isn’t correct. To prevent this, I created a GitHub Workflow that will validate the Bicep and the Bicepparam files when a pull request is created or updated. This way, I can’t forget to update the Bicepparam files.

The Bicep Linter GitHub Workflow file

name: 🎗️ Bicep Linter

on:
  pull_request:
    types: [opened, reopened, synchronize, edited, ready_for_review]

  workflow_dispatch:

permissions:
  id-token: write
  contents: write
  actions: read

concurrency:
  group: $-$
  cancel-in-progress: true

jobs:
  lint:
    name: Lint Bicep and Bicepparam files
    environment: AZURE
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Lint Bicep and Bicepparam files
        run: |
          az bicep install
          success="true"

          CHANGED_FOLDERS=$(git diff HEAD~1 --name-only | xargs dirname)
          echo "📂 Changed folders:"
          echo $CHANGED_FOLDERS

          FILES=$(find $CHANGED_FOLDERS \( -name "*.bicep" -o -name "*.bicepparam" \))
          echo "🔍 Files to lint:"
          echo $FILES

          for file in $FILES; do
          { # 'try' block
            echo "🎗️ Linting $file" && az bicep lint --file $file
          } || { # 'catch' block
            success="false"
          }
          done

          if [ "$success" = "false" ]; then
            echo "::error::❌ Linting of the Bicep files failed."
            exit 1;
          fi;

Checkout code

To only Lint the files that has been changed, and not Lint the entire repository, I use the actions/checkout@v4 action with the fetch-depth: 0 parameter to fetch all history for all branches and tags. This way, the git diff command can be used to get the changed files.

Get changed files

With the git diff HEAD~1 --name-only command, I get all the changed files in the pull request. The --name-only parameter will only return the file names and not the content of the files. The HEAD~1 parameter will compare the changed files with the main branch. This way, only the changed files in the pull request will be returned.

Get the changed folder

Now we only have the files that has been changed, with the xargs dirname command, we can get the folders of the changed files. Because the Bicep file has change, the Bicepparam files in the same folder should be validated as well.

Find the Bicep and Bicepparam files

With the find $CHANGED_FOLDERS \( -name "*.bicep" -o -name "*.bicepparam" \) command we get all the Bicep and Bicepparam files in the folders that has been changed. The -o parameter is a logical OR, so we get all the files that ends with .bicep or .bicepparam.

Lint the Bicep and Bicepparam files

Let’s Lint the files! With a for loop, we can loop through all the files and run the az bicep lint --file $file command. The --file parameter is the file that needs to be linted.

To create an App Services Managed Certificate there are two ways to create a certificate with Bicep. One for a apex domain and one for an subdomain. The validation of the ownership of the domain is the main difference. To generate a certificate the certificate authority would like to validate that the domain you try to get a certificate for is yours. That you are the owner of that (sub)domain name.

As a prerequisite you need to have an App Service Plan and an App Service or Function App with a custom domain ready. The Web App does not have a valid certificate already configured.

Create a Host Name binding without a certificate

param customHostname string = 'www.robertdeveen.com'
param webAppName string = 'robertdeveen'

resource hostNameBindingWithoutCertificate 'Microsoft.Web/sites/hostNameBindings@2022-03-01' = {
  name: '${webAppName}/${customHostname}'
  properties: {
    siteName: webAppName
    hostNameType: 'Verified'
    azureResourceType: 'Website'
    azureResourceName: webAppName
    customHostNameDnsRecordType: 'CName'
    sslState: 'Disabled'
  }
}

Create a App Services Managed Certificate for a subdomain

Create a CName record

In the DNS Zone, create a CName record pointing to *.azurewebsite.net. This is needed for the validation of the ownership of the domain.

Create a App Services Managed Certificate

param canonicalName string = 'www.robertdeveen.com'
param appServicePlanName string = 'robertdeveen-plan'
param location string = 'westeurope'

resource appServicePlan 'Microsoft.Web/serverfarms@2022-03-01' existing =  {
  name: appServicePlanName
}

resource certificate 'Microsoft.Web/certificates@2022-03-01' = {
  name: '${canonicalName}-certificate'
  location: location
  properties: {
    serverFarmId: appServicePlanResourceId
    // domainValidationMethod: 'cname-delegation' // or 'http-token'
    hostNames: [ canonicalName ]
    canonicalName: canonicalName
  }
}

Note: The documentation is not clear about the meaning of the domainValidationMethod field, it is a string. But the value that is accepted should be cname-delegation or http-token. Other values give the error message: “The parameter Properties.DomainValidationMethod has an invalid value.” The value cname-delegation is the only one working these days. The value http-token is not working anymore and just waiting a long time to end. The best solution is to not add that field.

Create a App Services Managed Certificate for an apex domain

Create a A record

In the DNS Zone, create an A records pointing to the IP address of the webapp. This is needed for the validation of the ownership of the domain.

param canonicalName string = 'robertdeveen.com'
param location string = 'westeurope'

resource nakedCertificate 'Microsoft.Web/certificates@2022-09-01' = {
  name: '${canonicalName}-certificate'
  location: location
  properties: {
    serverFarmId: appServicePlan.id
    canonicalName: canonicalName
  }
}

Create a App Services Managed Certificate for an apex and subdomain

THIS DOESN’T WORK! The documentation is not clear about this, but it is not possible to create a certificate for an apex and subdomain at the same time. You need to create two certificates, one for the apex and one for the subdomain.

// param canonicalName string = 'robertdeveen.com'
// param hostNames = [ 'www.robertdeveen.com', 'robertdeveen.com' ]
// param location string = 'westeurope'

// resource nakedCertificate 'Microsoft.Web/certificates@2022-09-01' = {
//   name: '${canonicalName}-certificate'
//   location: location
//   properties: {
//     serverFarmId: appServicePlan.id
//     hostNames: [ hostNames ]
//     canonicalName: canonicalName
//   }
// }

Bind certificate to the host name

We need to use a module to enable the certificate on the host name, as Bicep/ARM forbids using resource with this same type-name combination twice in one deployment.

module hostnameBindingWithCertificate './modules/hostname-binding.Bicep' = {
  name: '${customHostname}-hostnamebinding'
  params: {
    certificateThumbprint: certificate.outputs.thumbprint
    customHostname: customHostname
    webAppName: webAppName
  }
}

./module/hostname-binding.Bicep

resource hostNameBindingWithCertificate 'Microsoft.Web/sites/hostNameBindings@2022-03-01' = {
  name: '${webAppName}/${customHostname}'
  properties: {
    siteName: webAppName
    sslState: 'SniEnabled'
    thumbprint: certificateThumbprint
  }
}

Create an Azure Event Grid Namespace with MQTT enabled

First we need to create a new Azure Event Grid namespace or use an existing one and enable MQTT on the namespace. See the Azure documentation for more information on how to do this.

The broker address can be found in the Azure Portal, go to the “Overview” page of the Event Grid Namespace and find the MQTT Hostname. Remember this hostname, we need it later when we configure the MQTT integration in Home Assistant.

Create a client in Azure Event Grid MQTT

Then we need to create a client in Azure Event Grid MQTT. This can be done using the Azure Portal or with the Azure CLI. How this can be done is described in the Azure documentation. Make sure that you select “Thumbprint Match” as the authentication type when you use a self-signed certificate. Remember the Client Authentication Name. We need it later when we configure the MQTT integration in Home Assistant.

Create a Topic Space

Create a new Topic Space with a topic name of your choice. Add a Topic Template and enable all topics by entering # as a template.

Create Permission bindings

Now we need to allow all clients to publish and subscribe to the topics in the Topic Space we just created. This can be done by creating a new Permission binding. Create a new Permission binding for the Publisher and Subscriber operations for “Client group name” $all. This will allow all clients to publish and subscribe to all topics in the Topic Space.

Home Assistant configuration

To enable Home Assistant to connect to the Azure Event Grid MQTT broker we need to configure the MQTT integration in Home Assistant. Because we need to set some advanced settings in the MQTT integration, we need to enable the Advanced Mode in the Profile page of your account.

Create a new MQTT Integration

Create a new MQTT integration in Home Assistant and configure it to connect to the Azure Event Grid MQTT broker.

Fill in the MQTT Hostname in the broker address field. Set the port number to 8883 and set the Username to the one created earlier. The Password filed can be left empty.

Enable “Advanced options” and click Submit.

Enable “Use a client certificate” and click Submit.

Upload the client certificate and private key that you created earlier.

Set “Broker certificate validation” to “Auto”. You can leave the “CA certificate” field empty.

Enable “Ignore broker certificate validation”.

Set MQTT protocol version to 5 and transport type to TCP. And click Submit.

Disable will message

On the next page, disable the “Enable will message” option. This doesn’t work with the Azure Event Grid MQTT broker. You can choose to disable the “Enable birth message” option as well.

Test the MQTT connection

You can test the MQTT connection on the MQTT configuration page. First start listening to the Test topic. Then publish a message to the Test topic. You should see the message appear in the putput.

That’s all folks. You can now use the MQTT integration in Home Assistant to subscribe to events from Azure Event Grid MQTT broker.

© Robert de Veen

A composite of 70 photos from the dark sky above the Utrechtse Heuvelrug shot directly to the North. In the middle, you see the pole star and all the other stars dancing around. The straight lines are airplanes flying by and sending their Morse code to the audience on the ground. The photos were taken at 24mm, f/2.8, ISO 100, and 30 seconds. The photos were stacked and edited in Photoshop. The final image was processed in Adobe Lightroom.

Or as ChatGPT would say:

“The composite image capturing the dark sky above the Utrechtse Heuvelrug is a testament to the boundless wonders of the universe. It combines the natural splendor of star trails with the fleeting messages transmitted by airplanes, creating a poetic narrative etched across the canvas of the night sky. Through technical precision and creative vision, the photographer has gifted us a glimpse into the awe-inspiring beauty that lies just beyond our reach. So, next time you find yourself beneath a starlit sky, take a moment to appreciate the celestial symphony unfolding above, and perhaps, you too will capture your own unique piece of stardust.”

Author: OpenAI’s ChatGPT

© Robert de Veen

A composite of 50 photos from the dark sky above the Utrechtse Heuvelrug. The photos were taken at 24mm, f/2.8, ISO 100 and 30 seconds. The photos were stacked and edited in Photoshop. The final image was processed in Adobe Lightroom.

The man standing in the dark, that is me. I was standing there for 25 minutes in the cold and the dark. I was looking at the stars and the airplanes flying by. I was thinking about the universe and the meaning of life. No kidding, I was standing there just for 1 or 2 photo’s and then moved back out of the frame.