The new feature shown in this post was built by JasperFx Software as part of a client engagement. This is exactly the kind of novel or challenging issue we frequently help our clients solve. If there’s something in your shop’s ongoing efforts where you could use some extra technical help, reach out to sales@jasperfx.net and we’ll be happy to talk with you.
Wolverine 3.4 was released today with a large new feature for multi-tenancy through asynchronous messaging. This feature set was envisioned for usage in an IoT system using the full “Critter Stack” (Marten and Wolverine) where “our system” is centralized in the cloud, but has to communicate asynchronously with physical devices deployed at different client sites:
The system in question already uses Marten’s support for separating per tenant information into separate PostgreSQL databases. Wolverine itself works with Marten’s multi-tenancy to make that a seamless process within Wolverine messaging workflows. All of that arguably quite robust already support was envisioned to be running within either HTTP web services or asynchronous messaging workflows completely controlled by the deployed application and its peer services. What’s new with Wolverine 3.4 is the ability to isolate the communication with remote client (tenant) devices and the centralized, cloud deployed “our system.”
We can isolate the traffic between each client site and our system first by using a separate Rabbit MQ broker or at least a separate virtual host per tenant as implied in the code sample from the docs below:
var builder = Host.CreateApplicationBuilder();
builder.UseWolverine(opts =>
{
// At this point, you still have to have a *default* broker connection to be used for
// messaging.
opts.UseRabbitMq(new Uri(builder.Configuration.GetConnectionString("main")))
// This will be respected across *all* the tenant specific
// virtual hosts and separate broker connections
.AutoProvision()
// This is the default, if there is no tenant id on an outgoing message,
// use the default broker
.TenantIdBehavior(TenantedIdBehavior.FallbackToDefault)
// Or tell Wolverine instead to just quietly ignore messages sent
// to unrecognized tenant ids
.TenantIdBehavior(TenantedIdBehavior.IgnoreUnknownTenants)
// Or be draconian and make Wolverine assert and throw an exception
// if an outgoing message does not have a tenant id
.TenantIdBehavior(TenantedIdBehavior.TenantIdRequired)
// Add specific tenants for separate virtual host names
// on the same broker as the default connection
.AddTenant("one", "vh1")
.AddTenant("two", "vh2")
.AddTenant("three", "vh3")
// Or, you can add a broker connection to something completel
// different for a tenant
.AddTenant("four", new Uri(builder.Configuration.GetConnectionString("rabbit_four")));
// This Wolverine application would be listening to a queue
// named "incoming" on all virtual hosts and/or tenant specific message
// brokers
opts.ListenToRabbitQueue("incoming");
opts.ListenToRabbitQueue("incoming_global")
// This opts this queue out from being per-tenant, such that
// there will only be the single "incoming_global" queue for the default
// broker connection
.GlobalListener();
// More on this in the docs....
opts.PublishMessage<Message1>()
.ToRabbitQueue("outgoing").GlobalSender();
});
With this solution, we now have a “global” Rabbit MQ broker we can use for all internal communication or queueing within “our system”, and a separate Rabbit MQ virtual host for each tenant. At runtime, when a message tagged with a tenant id is published out of “our system” to a “per tenant” queue or exchange, Wolverine is able to route it to the correct virtual host for that tenant id. Likewise, Wolverine is listening to the queue named “incoming” on each virtual host (plus the global one), and automatically tags messages coming from the per tenant virtual host queues with the correct tenant id to facilitate the full Marten/Wolverine workflow downstream as the incoming messages are handled.
Now, let’s switch it up and use Azure Service Bus instead to basically do the same thing. This time though, we can register additional tenants to use a separate Azure Service Bus fully qualified namespace or connection string:
var builder = Host.CreateApplicationBuilder();
builder.UseWolverine(opts =>
{
// One way or another, you're probably pulling the Azure Service Bus
// connection string out of configuration
var azureServiceBusConnectionString = builder
.Configuration
.GetConnectionString("azure-service-bus");
// Connect to the broker in the simplest possible way
opts.UseAzureServiceBus(azureServiceBusConnectionString)
// This is the default, if there is no tenant id on an outgoing message,
// use the default broker
.TenantIdBehavior(TenantedIdBehavior.FallbackToDefault)
// Or tell Wolverine instead to just quietly ignore messages sent
// to unrecognized tenant ids
.TenantIdBehavior(TenantedIdBehavior.IgnoreUnknownTenants)
// Or be draconian and make Wolverine assert and throw an exception
// if an outgoing message does not have a tenant id
.TenantIdBehavior(TenantedIdBehavior.TenantIdRequired)
// Add new tenants by registering the tenant id and a separate fully qualified namespace
// to a different Azure Service Bus connection
.AddTenantByNamespace("one", builder.Configuration.GetValue<string>("asb_ns_one"))
.AddTenantByNamespace("two", builder.Configuration.GetValue<string>("asb_ns_two"))
.AddTenantByNamespace("three", builder.Configuration.GetValue<string>("asb_ns_three"))
// OR, instead, add tenants by registering the tenant id and a separate connection string
// to a different Azure Service Bus connection
.AddTenantByConnectionString("four", builder.Configuration.GetConnectionString("asb_four"))
.AddTenantByConnectionString("five", builder.Configuration.GetConnectionString("asb_five"))
.AddTenantByConnectionString("six", builder.Configuration.GetConnectionString("asb_six"));
// This Wolverine application would be listening to a queue
// named "incoming" on all Azure Service Bus connections, including the default
opts.ListenToAzureServiceBusQueue("incoming");
// This Wolverine application would listen to a single queue
// at the default connection regardless of tenant
opts.ListenToAzureServiceBusQueue("incoming_global")
.GlobalListener();
// Likewise, you can override the queue, subscription, and topic behavior
// to be "global" for all tenants with this syntax:
opts.PublishMessage<Message1>()
.ToAzureServiceBusQueue("message1")
.GlobalSender();
opts.PublishMessage<Message2>()
.ToAzureServiceBusTopic("message2")
.GlobalSender();
});
This is a lot to take in, but the major point is to keep client messages completely separate from each other while also enabling the seamless usage of multi-tenanted workflows all the way through the Wolverine & Marten pipeline. As we deal with the inevitable teething pains, the hope is that the behavioral code within the Wolverine message handlers never has to be concerned with any kind of per-tenant bookkeeping. For more information, see:
- Multi-Tenancy with Database per Tenant from the Marten documentation
- Multi-Tenancy with Wolverine
- Multi-Tenancy and Marten from the Wolverine documentation
- Multi-Tenancy with Rabbit MQ from the Wolverine documentation
- Multi-Tenancy with Azure Service Bus from the Wolverine documentation
And as I typed all of that out, I do fully realize that there would be some value in having a comprehensive “Multi-Tenancy with the Critter Stack” guide in one place.
Summary
I honestly don’t know if this feature set gets a lot of usage, but it came out of what’s been a very productive collaboration with JasperFx’s original customer as we’ve worked together on their IoT system. Quite a bit of improvements to Wolverine have come about as a direct reaction to friction or opportunities that we’ve spotted with our collaboration.
As far as multi-tenancy goes, I think the challenges for the Critter Stack toolset has been to give our users all the power they need to keep data and now messaging completely separate across tenants while relentlessly removing repetitive code ceremony or usability issues. My personal philosophy is that lower ceremony code is an important enabler of successful software development efforts over time.
The Shade Tree Developer 