Handling Azure Servicebus messages with WebJob SDK 3.x

A lot of the applications that I deploy to Azure are deployed as an Azure AppService. Most people use the AppService as a web application or web API host. But the AppService is more than a collection of HTTP endpoints. It also allows to host jobs. These jobs are background processes which are ideal for things that need to run once every night or for handling Azure Service Bus messages. In this post I will describe my experience deploying .net core 3.1 webjob using the WebJob 3.x SDK, for listening to messages being sent to an Azure Service Bus.

Start with the basics

Let’s start with creating a new command line project. As said I’m using .net core 3.1 which is currently the latest .net core version. We change the Program class that is created by VisualStudio to look like the code below:

using System;
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;

namespace Cbx.MessageHandlingJob
{
    class Program
    {
        public static async Task Main(string[] args)
        {
            try
            {
                var hostBuilder = new HostBuilder();
                hostBuilder
                    .ConfigureWebJobs((context, builder) =>
                    {
                        builder.AddAzureStorageCoreServices();
                        builder.AddServiceBus(options =>
                            {
                                options.ConnectionString = "your azure service bus connection string here";
                                options.MessageHandlerOptions.AutoComplete = true;
                            });
                    });
                var host = hostBuilder.Build();
                using (host)
                {
                    await host.RunAsync();
                }
            }
            catch (Exception e)
            {
                Console.WriteLine($"Web job failed: {e}");
                throw;
            }
        }
    }
}

As you can see, we have a fairly simple command line application. The special thing about this (but still very standard web jobs 3.x) is that we create a host using the HostBuilder. In the ConfigureWebJobs extension method we tell the host it should connect to the service bus.
First we set the connection string to the service bus. You can find your service bus connection string in the Azure portal.
The MessageHandlerOptions property allow you to set the way messages are handled. By setting AutoComplete to true, messages will be retried when at any point an exception is thrown and not handled.

Message trigger

To receive messages from the service bus, simply add a class to your project. In the class, create a method with a parameter of type Microsoft.Azure.ServiceBus.Message. On that message parameter, you need to add an attribute Microsoft.Azure.WebJobs.ServiceBusTrigger.

using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;

namespace Cbx.MessageHandlingJob
{
    public class MessageReceiver
    {
        private readonly ILogger<MessageReceiver> _logger;

        public MessageReceiver(ILogger<MessageReceiver> logger)
        {
            _logger = logger;
        }

        public Task HandleMessage([ServiceBusTrigger("yourTopic", "yourSubscription")] Message message)
        {
            _logger.LogInformation("Received message with ID {0}.", message.MessageId);
            return Task.CompletedTask;
        }
    }
}

The trigger will be picked up by the AddServiceBus in our Program.Main method and once the web job has started correctly, all messages on the service bus subscription will be sent to the MessageReceiver.HandleMessage method.

How to deploy

The reason I created this post is because I had a hard time figuring out how to deploy the web job correctly for handling messages. In the documentation I found, scattered across several pages, I figured out you need to deploy your web job project to the folder \App_Data\jobs\{type of web job}\{name of the web job}. The { name of the web job} is whatever name you want to give your job. The {type of web job} can be replaced by “triggered” or “continuous”. Whether to take triggered or continuous is not just some random decision, it impacts the way your job will be executed.

When first trying this out, I figured that since I’m using a trigger attribute, that the job is a triggered job. I was wrong. Using the ServiceBusTrigger doesn’t mean you need to deploy your job as a triggered job, you need to deploy it as a continuous job.

Looking back it’s quite obvious. Since listening to messages is something you need to do continuously, it makes sense to publish the job as a continuous web job. Unfortunately it took me some time to figure that out.

Configuration issues

Apparently there are some things you need to know to get the show on the road. Things I did not find in the Microsoft documentation.
One of them is a connection string to an Azure Storage Account. This storage account will be used to write job status and when you require a singleton, it will also be used for locking. So when you get a strange exception saying you didn’t provide a connection string, add this to your configuration file or user secrets:

{
  "ConnectionStrings": {
    "Storage": "UseDevelopmentStorage=true",
    "AzureWebJobsDashboard": "UseDevelopmentStorage=true"
  }
}

The “Storage” connection string is a connection string to Azure Blob Storage, used to write the status and to allow webjobs to run as singletons. When a job is required to run as a singleton, the job will create some locking files in the blob storage.
The “AzureWebJobsDashboard” connection string is also a connection string to Azure Blob Storage. It is used by web jobs to write status info. That status info is accessible via the Kudu of your WebApp. I’m not going to explain in detail what Kudu does or where you can find it (there is lots of info online on that), but sufficient to say via that dashboard you’re able to see when your job runs and what the console output of the job is.

What’s next

In the current implementation I’ve shown above, I use a property of type Message. According to the documentation it should be possible to use a typed message. So when the type of the message on your service bus is of type MyCustomMessage, it should be possible to use a parameter of type MyCustomMessage instead of Message. That way you don’t need to do serialization of the message body, but use the deserialized type directly. In my case, several types of messages are serialized on the Azure Service Bus and the way the messages are serialized is quite custom, so using the Message property made more sense.

Conclusion

Using web jobs it is quite easy to handle messages from a service bus (once you figure out how it works). You can easily connect to the right service bus and you can easily access the message and all of it’s attributes.