Tag Archives: FubuMVC

Health Monitoring and Task Reassignment in our Service Bus Applications

FubuMVC 3.0 actually has a full blown service bus framework that started as an add on project called “FubuTransportation.” We’ve used it in production for 3 years, we’re generally happy with it, and it’s the main reason why we’ve done an about face and decided to continue with FubuMVC again.

Corey Kaylor and I have been actively working on FubuMVC again. We’re still planning a reboot of at least the service bus functionality to the CoreCLR with a more efficient architecture next year (“Jasper“), but for right now we’re just working to improve the performance and reliability of our existing service bus applications. The “health monitoring” and persistent task functionality explained here has been in our codebase for a couple years and used a little bit in production, but we’re about to try to use it for something mission critical for the first time. I’d love to have any feedback or suggestions for improvements you might have. All the code shown here is pulled from this namespace in GitHub.

A Distributed System Spread Over Several Nodes

For the sake of both reliability and the potential for horizontal scaling later, we want to be able to deploy multiple instances of our distributed application to different servers (or separate processes on the same box as shown below:

 

BasicApp

A distributed application behind a load balancer

We generally employ hardware load balancers to distribute incoming requests through all the available nodes. So far, all of this is pretty typical and relatively straight forward as long as any node can service any request.

However, what if your architecture includes some kind of stateful “agent” that can, or at least should, be active on only one of the nodes at a time?

I’m hesitant to describe what we’re doing as Agent Oriented Programming, but that’s what I’m drawing on to think through this a little bit.

Agents

“Agent” worker processes should only be running on a single node

In our case, we’re working with a system that is constantly updating a “grid” of information stored in memory and directing work through our call centers. Needless to say, it’s a mission critical process. What we’re attempting to do is to make the active “agent” for that planning grid be managed by FubuMVC’s service bus functionality so that it’s always running on exactly one node in the cluster. That means that we need to be able to:

  • Have the various nodes constantly checking up on each other to make sure that agent is running somewhere and the assigned node is actually up and responsive
  • Be able to initiate the assignment of that agent to a new node if it is not running at all
  • Potentially shut down any extraneous instances of that agent if there is more than one running

Years ago, Chris Patterson of MassTransit fame explained something to me called the Bully Algorithm that can be used for exactly this kind of scenario. With a lot of help from my colleague Ryan Hauert, we came up with the approach described in this post.

 

Persistent Tasks

I reserve the right to change the name later (IAgent maybe?), but for now the key interface for one of these sticky agents is shown below:

public interface IPersistentTask
{
    Uri Subject { get; }

    // This is supposed to be the health check
    // Should throw an exception if anything is wrong;)
    void AssertAvailable();
    void Activate();
    void Deactivate();
    bool IsActive { get; }

    // This method would perform the actual assignment
    Task<ITransportPeer> SelectOwner(IEnumerable<ITransportPeer> peers);
}

Hopefully the interface is largely self descriptive. We were already using Uri’s throughout the rest of the  code, and that made sense to us to use that to identify the persistent tasks. This interface gives developers the hooks to start or stop the task from running, a way to do health checks, and a way to apply whatever kind of custom owner selection algorithm you want.

These persistent tasks are added to a FubuMVC application by registering an instance of this interface into the application container (there is a simple recipe for standalone tasks that deals with both interfaces in one class):

public interface IPersistentTaskSource
{
    // The scheme or protocol from the task Uri's
    string Protocol { get; }

    // Subjects of all the tasks built by this
    // object that should be running
    IEnumerable<Uri> PermanentTasks();

    // Create a task object for the given subject
    IPersistentTask CreateTask(Uri uri);
}

The IPersistentTaskSource might end up going away as unnecessary complexity in favor of just directly registering IPersistentTask’s. It was built with the idea of running, assigning, and monitoring agents per customer/tenant/region/etc. I’ve built a couple systems in the past half decade where it would have been very advantageous to have had that functionality.

The ITransportPeer interface used in the SelectOwner() method models the available nodes and it’s described in the next section.

 

Modeling the Nodes

The available nodes are modeled by the ITransportPeer shown below:

public interface ITransportPeer
{
        // Try to make this node take ownership of a task
	Task<OwnershipStatus> TakeOwnership(Uri subject);

        // Tries to ask the peer what the status is for all
        // of its assigned tasks
	Task<TaskHealthResponse> CheckStatusOfOwnedTasks();

	void RemoveOwnershipFromNode(IEnumerable<Uri> subjects);

	IEnumerable<Uri> CurrentlyOwnedSubjects();

	string NodeId { get; }
	string MachineName { get; }
	Uri ControlChannel { get; }

        // Shutdown a running task
	Task<bool> Deactivate(Uri subject);
}

ITransportPeer’s come in just two flavors:

  1. A class called PersistentTaskController that directly controls and manages the tasks on the executing node.
  2. A class called TransportPeer that represents one of the external nodes. The methods in this version send messages to the control channel of the node represented by the peer object and wait for a matching response. The other nodes will consume those messages and make the right calls on the local PersistentTaskController.

 

Reassigning Tasks

Now that we have a way to hook in tasks and a way to model the available peers, we need some kind of mechanism within IPersistentTask classes to execute the reassignment. Right now, the only thing we’ve built and used so far is a simple algorithm to assign a task based on an order of preference using the OrderedAssignment class shown below:

public class OrderedAssignment
{
	private readonly Uri _subject;
	private readonly ITransportPeer[] _peers;
	private int _index;

	public OrderedAssignment(Uri subject, IEnumerable<ITransportPeer> peers)
	{
		_subject = subject;
		_peers = peers.ToArray();
		_index = 0;
	}

	public async Task<ITransportPeer> SelectOwner()
	{
		return await tryToSelect().ConfigureAwait(false);
	}

	private async Task<ITransportPeer> tryToSelect()
	{
		var transportPeer = _peers[_index++];

		try
		{
			var status = await transportPeer.TakeOwnership(_subject).ConfigureAwait(false);

			if (status == OwnershipStatus.AlreadyOwned || status == OwnershipStatus.OwnershipActivated)
			{
				return transportPeer;
			}
		}
		catch (Exception e)
		{
			Debug.WriteLine(e);
		}

		if (_index >= _peers.Length) return null;

		return await tryToSelect().ConfigureAwait(false);
	}
}

Inside of an IPersistentTask class, the ordered assignment could be used something like this:

public virtual Task<ITransportPeer> SelectOwner(IEnumerable<ITransportPeer> peers)
{
    // it's lame, but just order by the control channel Uri
    var ordered = peers.OrderBy(x => x.ControlChannel.ToString());
    var completion = new OrderedAssignment(Subject, ordered);

    return completion.SelectOwner();
}

 

Health Monitoring via the Bully Algorithm

So now we have a way to model persistent tasks, reassign tasks, and model the connectivity to all the available nodes.

Inside of PersistentTaskController is this method that checks all the known persistent task state on every known running node:

public async Task EnsureTasksHaveOwnership()
{
	// Go run out and check the status of all the tasks that are
	// theoretically running on each node
	var healthChecks = AllPeers().Select(async x =>
	{
		var status = await x.CheckStatusOfOwnedTasks().ConfigureAwait(false);
		return new { Peer = x, Response = status };
	}).ToArray();

	var checks = await Task.WhenAll(healthChecks).ConfigureAwait(false);

	// Determine what corrective steps, if any, should be taken
        // to ensure that every known task is running in just one place
	var planner = new TaskHealthAssignmentPlanner(_permanentTasks);
	foreach (var check in checks)
	{
		planner.Add(check.Peer, check.Response);
	}

	var corrections = planner.ToCorrectionTasks(this);

	await Task.WhenAll(corrections).ConfigureAwait(false);

	_logger.Info(() => "Finished running task health monitoring on node " + NodeId);
}

In combination with the TaskHealthAssignmentPlanner class, this method is able to jumpstart any known tasks that either aren’t running or were running on a node that is no longer reachable or reports that its tasks are in an error state.

The EnsureTasksHaveOwnership() method is called from a system level polling job running in a FubuMVC application. There’s an important little twist on that though. To try to ensure that there’s much less chance of unpredictable behavior from the health monitoring checks running on each node simultaneously, the timing of the polling interval is randomized from this settings class:

public double Interval
{
    get
    {
        // The *first* execution of the health monitoring takes
        // place 100 ms after the app is initialized
        if (_initial)
        {
            _initial = false;
            return 100;
        }
                
        // After the first call, the polling interval is randomized
        // between each call
        return Random.Next(MinSeconds, MaxSeconds) * 1000;
    }
}

I found an article advising you to randomize the intervals somewhere online at the time we were building this two years ago, but I don’t remember where that was:(

By using the bully algorithm, we’re able to effectively make a cluster of related nodes able to check up on each other and start up or reassign any tasks that have gone down. We’re utilizing this first to do a “ready standby” failover of an existing system.

Actually Doing the Health Checks

The health check needs to run some kind of “heartbeat” action implemented through the IPersistentTask.AssertAvailable() method on each persistent task object to ensure that it’s really up and functioning. The following code is taken from PersistentTaskController where it does a health check on each running local task:

public async Task<TaskHealthResponse> CheckStatusOfOwnedTasks()
{
	// Figure out which tasks are running on this node right now
	var subjects = CurrentlyOwnedSubjects().ToArray();

	if (!subjects.Any())
	{
		return TaskHealthResponse.Empty();
	}

	// Check the status of each running task by calling the
	// IPersistentTask.AssertAvailable() method
	var checks = subjects
		.Select(async subject =>
		{
			var status = await CheckStatus(subject).ConfigureAwait(false);
			
			return new PersistentTaskStatus(subject, status);
		})
		.ToArray();

	var statusList = await Task.WhenAll(checks).ConfigureAwait(false);

	return new TaskHealthResponse
	{
		Tasks = statusList.ToArray()
	};
}

public async Task<HealthStatus> CheckStatus(Uri subject)
{
	var agent = _agents[subject];

	return agent == null 
		? HealthStatus.Unknown 
		: await checkStatus(agent).ConfigureAwait(false);
}

private static async Task<HealthStatus> checkStatus(IPersistentTaskAgent agent)
{
	return agent.IsActive
		? await agent.AssertAvailable().ConfigureAwait(false)
		: HealthStatus.Inactive;
}

 

 

Subscription Storage

Another obvious challenge is how does each node “know” about its peers? FubuMVC pulls that off with its “subscription” subsystem. In our case, each node is writing information about itself to a shared persistence store (mostly backed by RavenDb in our ecosystem, but we’re moving that to Marten). The subscription persistence also enables each node to discover its peers.

Once the subscriptions are established, each node can communicate with all of its peers through the control channel addresses in the subscription storage. That basic architecture is shown below with the obligatory boxes and arrows diagram:

Subscriptions

The subscription storage was originally written to enable dynamic message subscriptions between systems, but it’s also enabled our health monitoring strategy shown in this post.

 

 

Control Queue

We need the health monitoring and subscription messages between the various nodes to be fast and reliable. We don’t want the system level messages getting stuck in queues that might be backed up with normal activity. To that end, we finally put the idea of a designated “control channel” into FubuMVC so that you can designate a single channel as the preferred mechanism for sending control messages.

The syntax for making that designation is shown below in a code sample taken from FubuMVC’s integrated testing:

public ServiceRegistry()
{
    // The service bus functionality is "opt in"
    ServiceBus.Enable(true);

    // I explain what "Service" in the next code sample
    Channel(x => x.Service)
        // Listen for incoming messages on this channel
        .ReadIncoming()

        // Designate this channel as preferred for system level messages         
        .UseAsControlChannel()

        // Opts into LightningQueue's non-persistent mode               
        .DeliveryFastWithoutGuarantee(); 

    // I didn't want the health monitoring running on this node
    ServiceBus.HealthMonitoring
        .ScheduledExecution(ScheduledExecution.Disabled);
}

 

If you’re wondering what in the world “x => x.Service” refers to in the code above, that just ties into FubuMVC’s strong typed configuration support (effectively the same concept as the new IOptions configuration in ASP.Net Core, just with less cruft;)). The application described by ServiceRegistry shown above also includes a class that holds configuration items specific to this application:

public class TestBusSettings
{
    public Uri Service { get; set; } = "lq.tcp://localhost:2215/service".ToUri();
    public Uri Website { get; set; } = "lq.tcp://localhost:2216/website".ToUri();
}

The primary transport mechanism we use is LightningQueues (LQ), an OSS library built and maintained by my colleague Corey Kaylor. LQ is normally a “store and forward” queue, but it has a new, opt-in “non persistent” mode (like ZeroMQ, except .Net friendly) that we can exploit for our control channels in FubuMVC. In the case of the control queues, it’s advantageous to not persist those messages anyway.

 

My Concerns

It’s damn complicated and testing was obscenely hard. I’m a little worried about network hiccups causing it to unnecessarily try to reassign tasks. We might put some additional retries into the health checks. The central subscription persistence is a bit of a concern too because that’s a single point of failure.

Advertisements

Testing HTTP Handlers with No Web Server in Sight

FubuMVC 2.0 and 3.0 introduced some tooling I called “Scenarios” that allow users to write mostly declarative integration tests against the entire HTTP pipeline in memory without having to host the application in a web server. I promised a coworker that I would write a blog post about using Scenarios for an internal team that wants to start using it much more in their work. A week of procrastination later and here you go:

NOTE: All samples are using FubuMVC 3.0

Why Integration Tests?

From the very beginning, we tried very hard to make unit testing FubuMVC action methods in isolation as easy as possible. I think we largely succeeded in that goal. However, within the context of a handling an HTTP request, FubuMVC like most web frameworks will potentially wrap those action methods with various middleware strategies for cross cutting technical things like authentication, authorization, logging, transaction management, and content negotiation. At some point, to truly exercise an HTTP endpoint you really do need to write an integration test that exercises the entire chain of HTTP handlers for an HTTP request exactly the way it will be configured inside the running application.

Toward that end, I built a class called EndpointDriver in early versions of FubuMVC that you could use to write integration tests against a FubuMVC application hosted with an embedded Katana server. This early tooling just wrapped WebClient with a FubuMVC specific fluent interface for resolving url’s, setting common options like the content-type and accepts headers, and verifying parts of the HTTP response. Below is a sample from our content negotiation support integration tests in FubuMVC 1.3 (“endpoints” is a reference to the EndpointDriver object for the running application):

[Test]
public void force_to_json_with_querystring()
{
    endpoints.Get("conneg/override/Foo?Format=Json", acceptType: "text/html")
        .ContentTypeShouldBe(MimeType.Json)
        .ReadAsJson<OverriddenResponse>()
        .Name.ShouldEqual("Foo");
}

EndpointDriver was fine at first, but our test library started getting slower as we added more and more tests and the fluent interface just never kept up with everything we needed for HTTP testing (plus I think that WebClient is awkward to use).

Using OWIN for HTTP “Scenarios”

As part of my FubuMVC 2.0 effort last year, I knew that I wanted a much better mechanism than the older EndpointDriver for doing integration testing of HTTP endpoints. Specifically, I wanted:

  • To be able to run HTTP requests and verify the response without having to take the performance hit of a web server
  • To run a FubuMVC application as it would be configured in production
  • To completely configure any part of an HTTP request
  • To be able to declaratively express multiple assertions against the expected response
  • To utilize FubuMVC’s support for “reverse URL resolution” for more traceable tests
  • Access to the raw HTTP request and response for anything unusual you would need to do that didn’t have a specific helper

The end result was a mechanism I called “Scenario’s” that exploited FubuMVC’s OWIN support to run HTTP requests in memory using this signature off of the new FubuRuntime object I explained in an earlier blog post:

OwinHttpResponse Scenario(Action<Scenario> configuration)

The Scenario object models both the HTTP request provides a way to specify expectations about the HTTP response for commonly used things like HTTP status codes, header values, and checking for the presence of string values in the HTTP response body. If need be, you also have access to FubuMVC’s abstractions for the entire HTTP request and response (more on this later).

To make this concrete, let’s say that you’re working through a “Hello, World” exercise with FubuMVC with this class and action method that just returns the text “Hello, World” when you issue a GET to the root “/” url of an application:

public class HomeEndpoint
{
    public string Index()
    {
        return "Hello, World";
    }
}

A scenario test for the action above would look like this code below:

using (var runtime = FubuRuntime.Basic())
{
    // Execute the home route and verify
    // the response
    runtime.Scenario(_ =>
    {
        _.Get.Url("/");

        _.StatusCodeShouldBeOk();
        _.ContentShouldBe("Hello, World");
        _.ContentTypeShouldBe("text/plain");
    });
}

In the scenario above, I’m issuing a GET request to the “/” url of the application and specifying that the resulting status code should be HTTP 200, “content-type” response header should be “text/plain”, and the exact contents of the response body should be “Hello, World.” When a Scenario is executed, it will run every single assertion instead of quitting on the first failure and report on every failed expectation in the specification output. This behavior is valuable when you have to author specifications with slower running scenario setup.

Specifying Url’s

FubuMVC has a model for reverse URL lookup from any endpoint method or the input model that we exploited in Scenario’s for traceable tests:

host.Scenario(_ =>
{
    // Specify a GET request to the Url that runs an endpoint method:
    _.Get.Action<InMemoryEndpoint>(e => e.get_memory_hello());

    // Or specify a POST to the Url that would handle an input message:
    _.Post

        // This call serializes the input object to Json using the 
        // application's configured JSON serializer and setting
        // the contents on the Request body
        .Json(new HeaderInput {Key = "Foo", Value1 = "Bar"});

    // Or specify a GET by an input object to get the route parameters
    _.Get.Input(new InMemoryInput { Color = "Red" });
});

I like the reverse url lookup instead of specifying Url’s directly in the scenarios because:

  1. It makes your scenario tests traceable to the actual handling code
  2. It insulates your scenarios from changes to the Url structures later

Checking the Response Body

For the 3.0 work I did a couple months ago, I fleshed out the Scenario support with more mechanisms to analyze the HTTP response body:

host.Scenario(_ =>
{
    // set up a request here

    // Read the response body as text
    var bodyText = _.Response.Body.ReadAsText();

    // Read the response body by deserializing Json
    // into a .net type with the application's
    // configured Json serializer
    var output = _.Response.Body.ReadAsJson<MyResponse>();

    // If you absolutely have to work with Xml...
    var xml = _.Response.Body.ReadAsXml();
});

Some Other Things…

I’ll happily explain the details of this list on request, but here are some other attributes of Scenario’s that FubuMVC supports right now:

  • You can specify expected values for HTTP response headers
  • You can assert on status codes and descriptions
  • There are helpers to send Json or Xml serialized data based on an input object message
  • There is a mechanism that allows you to disable all security middleware in the application for a single Scenario that has been frequently helpful in testing
  • You have access to the underlying IoC container for the running application from the Scenario if you need to resolve and use application services
  • FubuMVC is now StructureMap 4.0-only for its IoC usage, so we’re able to rely on StructureMap’s child container feature to resolve services during a Scenario execution from a unique child container per run. This allows you to replace services in your application with fakes, mocks, and stubs in a way that prevents your fake services from impacting more than one test.

Scenarios in Jasper

If you didn’t see my blog post earlier this year, FubuMVC is getting a complete reboot into a new project called Jasper late this year/early next year. I absolutely plan on bringing the Scenario support forward into Jasper very early, but this time around we’re completely dropping all of FubuMVC’s HTTP abstractions in favor of directly using the OWIN environment dictionary as the single model of HTTP requests and responses. My thought right now is that we’ll invest heavily in extension methods hanging off of IDictionary<string, object> for commonly used operations against that OWIN dictionary.

To some extent, we’re hoping as well that there will be a good ecosystem of OWIN helpers from other people and projects that will be usable from within Jasper.

Other Reading

Transaction Scoping in FubuMVC with RavenDb and StructureMap

TL;DR – We built unit of work and transactional boundary mechanics into our RavenDb integration with FubuMVC (with StructureMap supplying the object scoping) that I’m still happy with and I’d consider using the same conceptual design again in the future.

Why write this now long after giving up on FubuMVC as an OSS project? We still use FubuMVC very heavily at work, I’m committed to supporting StructureMap as long as it’s relevant, and RavenDb is certainly going strong. I’ve seen other people trying to accomplish the same kind of IoC integration and object scoping that I’m showing here on other frameworks and this might be useful to those folks. Mostly though, this post might benefit some of our internal teams and almost every enterprise software project involves some kind of transaction management.

Transaction Boundaries

If you’re working on any kind of software system that persists state, you probably need to be consciously aware of the ACID rules (AtomicityConsistencyIsolationDurability) to determine where your transaction boundaries should be. Roughly speaking, you want to prevent any chance of a system getting into an invalid state because some database changes succeeded while other database changes within the same logical operation fail. The canonical example is transfering money between two different bank accounts. For that use case, you’re probably making at least three different changes to the underlying database:

  1. Subtract the amount of the transfer from the first account
  2. Add the amount of the transfer to the second account
  3. Write some sort of log record for the transfer

All three database operations above need to succeed together, or all three should be rolled back to prevent your system from getting into an invalid state. Failing to manage your transaction boundaries could result in the bank customer either losing some of their funds or the bank magically giving more money to the consumer. Following the ACID rules, we would make sure that all three operations are within the same database transaction so that they all have to succeed or fail together.

Using event sourcing and eventual consistency flips this on its head a little bit. In that case you might just capture the log record, then rebuild the “read side” views for the two accounts asynchronously to reflect the transfer. Needless to say, that can easily generate a different set of problems when there’s any kind of lag in the “eventual.”

The Unit of Work Pattern

Historically, one of the easier ways to collect and organize database operations into transactions has been the Unit of Work pattern that’s now largely built into most every persistence framework. In RavenDb, you use the IDocumentSession service as the logical unit of work. Using RavenDb, we can just collect all the documents that need to be persisted within a single transaction by calling the Delete() or Store() methods. When we’re ready to commit the entire unit of work as a transaction, we just call the SaveChanges() method to execute all our batched up persistence operations.

One of the big advantages to using the unit of work pattern is the ability to easily coordinate transaction boundaries across multiple unrelated classes, services, or functions — as long as each is working against the same unit of work object. The challenge then is to control the object lifecycle of the RavenDb IDocumentSession object in the lifecycle of a logical operation. While you can certainly use some sort of transaction controller (little ‘c’) to manage the lifecyle of an IDocumentSession object and coordinate its usage within the actual workers, we built IDocumentSession lifecycle handling directly into our FubuMVC.RavenDb library so that you can treat an HTTP POST, PUT, or DELETE in FubuMVC or the handling of a single message in the FubuTransportation service bus as a logical unit of work and simply let the framework itself do all your transaction boundary kind of bookkeeping for you.

IDocumentSession Lifecycle Management 

First off, FubuMVC utilizes the StructureMap Nested Container feature to create an isolated object scope per HTTP request — meaning that any type registered with the default lifecyle that is resolved by StructureMap within a request (or message) will be the same object instance. In the case of IDocumentSession, this scoping means that every concrete type built by StructureMap will have the same shared instance of IDocumentSession.

To make it concrete, let’s say that you have these three classes that all take in an IDocumentSession as a constructor argument:

    public class RavenUsingEndpoint
    {
        private readonly IDocumentSession _session;
        private readonly Worker1 _worker1;
        private readonly Lazy<Worker2> _worker2;

        public RavenUsingEndpoint(IDocumentSession session, Worker1 worker1, Lazy<Worker2> worker2)
        {
            _session = session;
            _worker1 = worker1;
            _worker2 = worker2;
        }

        public string post_session_is_the_same()
        {
            _session.ShouldBeTheSameAs(_worker1.Session);
            _session.ShouldBeTheSameAs(_worker2.Value.Session);

            return "All good!";
        }
    }

    public class Worker1
    {
        public IDocumentSession Session { get; set; }

        public Worker1(IDocumentSession session)
        {
            Session = session;
        }
    }

    public class Worker2 : Worker1
    {
        public Worker2(IDocumentSession session) : base(session)
        {
        }
    }

When a FubuMVC request executes that uses the RavenUsingEndpoint.post_session_is_the_same() method, FubuMVC is creating a RavenUsingEndpoint object, a Worker1 object, a Lazy<Worker2> object, and one single IDocumentSession object that is injected both into RavenUsingEndpoint and Worker1. Additionally, if the Lazy<Worker2> is evaluated, a Worker2 object will be created from the active nested container and the exact same IDocumentSession will again be injected into Worker2. With this type of scoping, we can use RavenUsingEndpoint, Worker1, and Worker2 in the same logical transaction without worrying about how we’re passing around IDocumentSession or dealing with any sort of less efficient transaction scoping tied to the thread.

TransactionalBehavior

So step 1, and I’d argue the hardest step, was making sure that the same IDocumentSession object was used by each handler in the request. Step 2 is to execute the transaction.

Utilizing FubuMVC’s Russian Doll model, we insert a new “behavior” into the request handling chain for simple transaction management:

    public class TransactionalBehavior : WrappingBehavior
    {
        private readonly ISessionBoundary _session;

        // ISessionBoundary is a FubuMVC.RavenDb service
        // that just tracks the current IDocumentSession
        public TransactionalBehavior(ISessionBoundary session)
        {
            _session = session;
        }

        protected override void invoke(Action action)
        {
            // The 'action' below is just executing
            // the handlers nested inside this behavior
            action();

            // Saves all changes *if* an IDocumentSession
            // object was created for this request
            _session.SaveChanges();
        }
    }

The sequence of events inside a FubuMVC request using the RavenDb unit of work mechanics would be to:

  1. Determine which chain of behaviors should be executed for the incoming url (routing)
  2. Create a new StructureMap nested container
  3. Using the new nested container, build the entire “chain” of nested behaviors
  4. The TransactionalBehavior starts by delegating to the inner behaviors, one of which would be…
  5. A behavior that invokes the RavenUsingEndpoint.post_session_is_the_same() method
  6. The TransactionBehavior object, assuming there are no exceptions, executes all the queued up persistence actions as a single transaction
  7. At the end of the request, FubuMVC will dispose the nested container for the request, which will also call Dispose() on any IDisposable object created by the nested container during the request, including the IDocumentSession object.

How does the TransactionalBehavior get applied?

In the early days we got slap happy with conventions and the simple inclusion of the FubuMVC.RavenDb library in the applications bin path was enough to add the TransactionBehavior to any route that responded to the PUT, POST, or DELETE http verbs. For FubuMVC 2.0 I made this behavior “opt in” instead of being automatic, so you just had to either include a policy to specify which routes or message handling chains needed the transactional semantics:

    public class NamedEntityRegistry : FubuRegistry
    {
        public NamedEntityRegistry()
        {
            Services(x => x.ReplaceService(new RavenDbSettings { RunInMemory = true}));

            // Applies the TransactionalBehavior to all PUT, POST, DELETE routes
            Policies.Local.Add<TransactionalBehaviorPolicy>();
        }
    }

You could also use attributes for fine grained application if there was a need for that.

Explicit Transaction Boundaries

I fully realize that many people will dislike the API usage with the nested closure below, but it’s worked out well for us in my opinion — especially when it’s rarely used as the exception case rather than the normal every day usage.

All the above stuff is great as long as your within the context of an HTTP request or a FubuTransportation message that represents a single logical transaction, but that leaves plenty of circumstances where you need to explicitly control transaction boundaries. To that end, we’ve used the ITransaction service in FubuPersistence to mark a logical transaction boundary by again executing a logical transaction within the scope of a completely isolated nested container for the operation.

Let’s say you have a simple transaction worker class like this one:

    public class ExplicitWorker
    {
        private readonly IDocumentSession _session;

        public ExplicitWorker(IDocumentSession session)
        {
            _session = session;
        }

        public void DoSomeWork()
        {
            // do something that results in writes
            // to IDocumentSession
        }
    }

To execute the ExplicitWorker.DoSomeWork() method in its own isolated transaction, I would invoke the ITransaction service like this:

// Just spinning up an empty FubuMVC application
// FubuMVC.RavenDb is picked up automatically by the assembly
// being in the bin path
using (var runtime = FubuApplication.DefaultPolicies().StructureMap().Bootstrap())
{
    // Pulling a new instance of the ITransaction service
    // from the main application container
    var transaction = runtime.Factory.Get<ITransaction>();
            
    transaction.Execute<ExplicitWorker>(x => x.DoSomeWork());
}

At runtime, the sequence of actions is:

  1. Create a new StructureMap nested container
  2. Resolve a new instance of ExplicitWorker from that nested container
  3. Execute the Action<ExplicitWorker> passed into the ITransaction against the newly created ExplicitWorker object to execute the logical unit of work
  4. Assuming that the action didn’t throw an exception, submit all changes to RavenDb as a single transaction
  5. Dispose the nested container

Use this strategy when you need to execute transactions outside the scope of FubuMVC HTTP requests or FubuTransportation message handling, or anytime when it’s appropriate to execute multiple transactions for a single HTTP request or FT message.