Tag Archives: Lamar

My OSS Plans for 2019

I wrote a similar post last year on My OSS Plans for 2018, and it wasn’t too far off what actually happened. I did switch jobs last May, and until recently, that dramatically slowed down my rate of OSS contributions and my ambition level for OSS work. I still enjoy building things, I’m in what’s supposed to be a mostly non-coding role now (but I absolutely still do), and working on OSS projects is just a good way to try to keep my technical skills up. So here we go:

Marten — I’m admittedly not super active with Marten these days as it’s a full fledged community driven project now. There has been a pile up of Linq related issues lately and the issue backlog is getting a bit big, so it really wouldn’t hurt for me to be more active. I happen to love Stephen King’s Dark Tower novels (don’t bother with the movie), but he used to complain that they were hard for him to get into the right head space to write more about Roland and his Ka-tet. That’s basically how I feel about the Linq provider in Marten, but it’s over due for some serious love.

Lamar (was BlueMilk) — Users are reporting some occasional memory usage problems that I think point to issues in Roslyn itself, but in the meantime I’ve got some workarounds in mind to at least alleviate the issue (maybe, hopefully, knock on wood). The one and only big feature idea for this year I have in mind is to finally do the “auto-factory” feature that I hope will knock out a series of user requests for more dynamic behavior.

Alba — I took some time at CodeMash this year and did a pretty substantial 3.0 release that repositioned Alba as a productivity helper on top of TestServer to hopefully eliminate a whole lot of compatibility issues with ASP.Net Core. I don’t know that I have any other plans for Alba this year, but it’s the one tool I work on that I actually get to use at work, so there might be some usability improvements over time.

Storyteller — I’d really love to sink into a pretty big rewrite of the user interface and some incremental — but perfectly backward compatible — improvements to the engine. After a couple years of mostly doing integration work, I suddenly have some UI centric projects ahead of me and I could definitely use some refresher on building web UIs. More on this in a later post.

Jasper — I think I’m finally on track for a 1.0 release in the next couple months, but I’m a little unsure if I want to wait for ASP.Net Core 3.0 or not. After that, it’s mostly going to be trying to build some community around Jasper. Most of my side project time and effort the past three years has been toward Jasper, I have conceptual notes on its architecture that go back at least 5 years, and counting its predecessor project FubuMVC, this has been a 10 year effort for me. Honestly, I think I’m going to finish the push to 1.0 just to feel some sense of completion.

Oakton — I feel like it’s basically done

StructureMap — I answer questions here and there, but otherwise it’s finished/abandoned

FubuMVC — Dead, but little bits and pieces of it live on in Oakton, Alba, and Jasper

 

Metaprogramming in C# with LamarCompiler

I gave a talk last week at NDC London called “Dynamic Runtime Code with Roslyn” that featured an example of using LamarCompiler and Lamar to create a minimal equivalent of the Refit library. If you’re not familiar with Refit, it allows you to express the intended service proxy to a web service with a .Net interface or abstract class, apply a few attributes to explain Url patterns or other HTTP request items, and then allow Refit to dynamically generate a class that implements your interface that deals with all the repetitive gunk of building Urls, serializing or deserializing JSON, and calling HttpClient to actually make the requests.

Cool, right? I’d argue that Refit is an example of metaprogramming inside of C# where a tool is able to generate and eventually execute code gleaned from data (an interface and a handful of attributes in this case) within the application. I tend to think of metaprogramming as something specific to the Ruby and other dynamic languages arena, but now that we have the ability to compile C# code on the fly with Roslyn, I think it’s going to become something much more commonly done in .Net systems.

Today in the Lamar codebase, there’s a little project called LamarRest that contains all the code I’m going to show here if you’re curious to see more about how it all fits together. I should say that I intended this code strictly as a demonstration of what LamarCompiler can do and do not have any plans right now to make this a production capable library.

The Problem and Conceptual Solution

Okay, let’s get concrete and write a very small web service that manages users. The whole API surface in this case is implemented by the UserController class shown below (the full source code for this web service is here):

    public class UserController : Controller
    {
        private readonly UserRepository _repository;
        private readonly ILogger _logger;

        public UserController(UserRepository repository, ILogger logger)
        {
            _repository = repository;
            _logger = logger;
        }

        [HttpPost("/user/create")]
        public IActionResult Create([FromBody] User user)
        {
            _logger.LogInformation($"I created a new user with the name '{user.Name}'");
            _repository.Users.Fill(user);
            return Ok();
        }
        
        [HttpGet("/user/{name}")]
        public User GetUser(string name)
        {
            return _repository.Users.FirstOrDefault(x => x.Name == name);
        }

        [HttpGet("/users")]
        public User[] GetUsers()
        {
            return _repository.Users.ToArray();
        }
    }

Now, back in the codebase for an application that needs to communicate with the user service, we ideally want to just write an interface to act as a proxy to the User web service that looks like this:

    public interface IUserService
    {
        [Get("/user/{name}")]
        Task GetUser(string name);

        [Post("/user/create")]
        Task Create(User user);
    }

In our application, we just want to interact with the IUserService and not really have to know where it comes from. To that end, I’m going to go a step farther than Refit does and say that our solution should automatically resolve the generated type for IUserService from our application’s IoC container on demand.

As you’ll notice, I did have to add a couple attributes ([Get] and [Post]) that just specify the Url pattern for the two proxy methods. In this case as well though, the presence of the route marker attributes tells the code we’re about to write that these methods should be generated to call HttpClient.

Implementing Our Refit Clone

To generate the concrete HTTP proxy classes, I’m going to leverage LamarCompiler’s Frame model. If you write out implementation of IUserService above, you’ll notice discrete set of logical steps, each of which becomes a LamarCompile Frame in this model:

  1. Resolving an HttpClient from the HttpClientFactory for our application.  That became BuildClientFrame.
  2. Fill the real Url by substituting any route arguments with method parameters from our interface method signatures. That’s implemented (crudely and incompletely) in LamarRest by the FillUrlFrame.
  3. If necessary, serialize the input boy argument to Json using Newtonsoft.Json’s default settings (Refit being a real library allows you to configure the Json serialization, LamarRest being a demo worked up the day before a conference talk, does not). For that purpose, I created SerializeJsonFrame.
  4. To actually send an Http request, we need to create an HttpRequestMessage object with the right Url, Http method, and the serialized Json if there is any. This one is a little easier, as I just subclass Lamar’s built in support for calling a class constructor with any mix of arguments and setters in the BuildRequestFrame class.
  5. To actually invoke HttpClient.SendAsync() , there’s just a built in MethodCall Frame inside of LamarCompiler.
  6. If there is a response body expected, as in the IUserService.GetUser() method, we need a Frame that can deserialize the contents of the HTTP response body into the desired model. I implemented that in DeserializeObjectFrame.

Okay, so that’s a lot of steps, but once these Frame classes were built, it was easy to assemble them to match an interface method. In LamarRest, there’s a fair sized class called GeneratedServiceType that handles all the code generation and compilation for these proxy classes. Most applicable to this discussion is the method that builds out the concrete proxy methods by assembling the necessary list of Frame objects as shown below:

public static void BuildOut(
    // The contract type
    Type interfaceType, 
    
    // The MethodInfo from Reflection that describes
    // the Url structure through attributes, the input type
    // if any, and the .Net type of the response (if any)
    MethodInfo definition, 
    
    // This models the method being generated by LamarCompiler
    GeneratedMethod generated)
{
    var path = definition.GetCustomAttribute();

    // Get the right HttpClient from IHttpClientFactory
    generated.Frames.Add(new BuildClientFrame(interfaceType));

    // Build out the Url from the method arguments and route
    // pattern
    var urlFrame = new FillUrlFrame(definition);
    generated.Frames.Add(urlFrame);

    
    // See if there is an input type that should be serialized
    // to the HTTP request body
    var inputType = DetermineRequestType(definition);
    if (inputType == null)
    {
        // Just build the HttpRequestMessage
        generated.Frames.Add(new BuildRequestFrame(path.Method, urlFrame, null));
    }
    else
    {
        // Add a step to serialize the input model to Json
        generated.Frames.Add(new SerializeJsonFrame(inputType));
        
        // Just build the HttpRequestMessage
        generated.Frames.Add(new BuildRequestFrame(path.Method, urlFrame, new SerializeJsonFrame(inputType)));
    }

    // Actually call the HttpClient to send the request
    generated.Frames.Call(x => x.SendAsync(null));

    // Is there a response type that should be serialized out of the HTTP
    // response body?
    var returnType = DetermineResponseType(definition);
    if (returnType != null)
    {
        // Deserialize the response JSON into a new variable
        var deserialize = new DeserializeObjectFrame(returnType);
        generated.Frames.Add(deserialize);
        
        // Return that deserialized object from the method
        generated.Frames.Return(deserialize.ReturnValue);
    }
}

I should note that LamarCompiler does some work of connecting variables, method arguments, and class fields by type and sometimes by name — which is why you see no explicit mention of how IHttpClientFactory comes into play. LamarCompiler is smart enough to see that a Frame object needs an IHttpClientFactory object, know that that service is a singleton-coped object in the application’s IoC container, and therefore generate a field for that service.

For the IUserService interface, LamarRest will generate this code for the concrete class, but I warn you now, it’s generated code and hence will be as ugly as sin:

    public class UserService : LamarRest.Testing.IUserService
    {
        private readonly System.Net.Http.IHttpClientFactory _httpClientFactory;

        public UserService(System.Net.Http.IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;
        }



        public async System.Threading.Tasks.Task GetUser(string name)
        {

            // From BuildClientFrame
            var httpClient = _httpClientFactory.CreateClient("IUserService");

            // From FillUrlFrame
            var url = $"/user/{name}";
            var httpRequestMessage = new System.Net.Http.HttpRequestMessage(new System.Net.Http.HttpMethod("GET"), url);
            using (var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage))
            {

                // From DeserializeObjectFrame
                var user = Newtonsoft.Json.JsonConvert.DeserializeObject(await httpResponseMessage.Content.ReadAsStringAsync());
                return user;
            }

        }


        public Task Create(UserApp.Controllers.User user)
        {

            // From BuildClientFrame
            var httpClient = _httpClientFactory.CreateClient("IUserService");

            // From FillUrlFrame
            var url = $"/user/create";

            // From SerializeJsonFrame
            var json = Newtonsoft.Json.JsonConvert.SerializeObject(user);
            var httpRequestMessage = new System.Net.Http.HttpRequestMessage(new System.Net.Http.HttpMethod("POST"), url){Content = new System.Net.Http.StringContent(json, System.Text.Encoding.UTF8, "application/json")};
            return httpClient.SendAsync(httpRequestMessage);
        }

Hooking it up to the Application

Lastly, we need to hook up our new LamarRest library to our application’s IoC container and application configuration. Assuming that you’re using Lamar as your IoC container, the configuration might be something like this:

            var container = new Container(x =>
            {
                x.AddHttpClient(
                    typeof(IUserService).Name, 
                    // You'd pull this from actual configuration in real life
                    c => c.BaseAddress = new Uri("http://localhost:5000"));
                
                // Adding a Lamar policy here that will auto-resolve LamarRest
                // proxies on demand
                x.Policies.Add<LamarRestPolicy>();
            });

In the code above, I’m heavily leveraging the standard ASP.Net Core AddHttpClient() registration to deal with base Urls and default HTTP request attributes like authentication credentials. I’d probably switch it to being strong typed later, but for now the convention is to name the HttpClient based on the interface name.

The second registration is Lamar specific to add a policy to allow Lamar to auto-resolve a concrete class for any interface that is otherwise unknown to the container and has methods marked with any kind of LamarRest route attribute.

Just for completeness sake, that’s shown below:

    public class LamarRestPolicy : Lamar.IFamilyPolicy
    {
        public ServiceFamily Build(Type type, ServiceGraph serviceGraph)
        {
            if (type.GetMethods().Any(x => x.HasAttribute()))
            {
                var rules = new GenerationRules("LamarRest");
                var generatedAssembly = new GeneratedAssembly(rules);
                var generatedType = new GeneratedServiceType(generatedAssembly, type);

                var container = (IContainer)serviceGraph.RootScope;
                
                container.CompileWithInlineServices(generatedAssembly);

                return new ServiceFamily(
                    type, 
                    new IDecoratorPolicy[0], 
                    new ConstructorInstance(type, 
                    generatedType.CompiledType, 
                    ServiceLifetime.Singleton
                ));
            }

            return null;
        }

The final result of all this setup is to be able to do this:

            
            // Build an implementation of this service as needed
            // I'm using service location here, but you'd be able
            // to inject IUserService as a constructor argument as well
            var userService = container.GetInstance<IUserService>();
            

 

Lamar 2.0 is Out! StructureMap’s Successor Grows up a bit

I was just able to publish Lamar 2.0 to Nuget and it looks like it’s indexed, so here we go!

Along with a whole lot of user reported bug fixes, the two big changes are:

  • We split the dynamic code compilation and code generation (the Frames model) into its own separate library called LamarCompiler so that interested folks could easily reuse that functionality without having the IoC support bundled along with it (even though you never needed to use the Lamar IoC code to take advantage of the code compilation but .Net devs are finicky about dependencies some times)
  • After a couple user requests for it, Lamar 2.0 supports optional and mandatory setter injection based on StructureMap’s setter injection support.\

Many thanks to all the folks that helped move this along by reporting GitHub issues and providing pretty damn useful reproduction steps. Special thanks to Mark Warpool for all his help in making Lamar go. Lamar, like Marten before it, has been a very positive experience for me in my dealings with the greater OSS community.

For more information, here’s:

 

What next for Lamar?

I don’t know when, but I’ll eventually get around to the “AutoFactory” feature, because I think that’s the single biggest blocker for many folks moving from StructureMap to Lamar. If LamarCompiler takes off at all, there’s worlds of opportunities to add new C# code constructs to the code generation model.

Lamar 1.0: Faster, modernized successor to StructureMap

EDIT: 6/15/2018: Fixed an erroneous link and don’t you know it, there’s already a Lamar 1.0.2 up on Nuget (might take a second for indexing to catch up first) with some fixes  to user reported problems.

Lamar is a new OSS library written in C# that is a new IoC container meant to replace the venerable, but increasingly creaky StructureMap library. Lamar is also Jasper‘s “special sauce” that provides runtime code generation and compilation that helps make Jasper’s runtime pipeline very efficient compared to its competitors. You can happily use the code generation and compilation without the IoC functionality. Thank you to Mark Warpool, Jordan Zaerr, Mark Wuhrith and many others for their help and early feedback.

I’m happy to announce that Lamar and Lamar.Microsoft.DependencyInjection (the ASP.Net Core adapter) were both published as v1.0.0 to Nuget this morning. It’s been beaten up a bit by early adopters and I finished off the last couple features around type scanning that I wanted for a v1 feature set, so here we go. I’m pretty serious about semantic versioning, so you can take this release as a statement that the public API signatures are stable. The documentation website is completely up to date with the v1 API and ready to go. If you’re kicking the tires on Lamar and run into any trouble, check out the Gitter room for Lamar.

To get going, check out Lamar’s Getting Started page.

Lamar as IoC Container

To get started, just add Lamar to your project through Nuget.

Most of the time you use an IoC container these days, it’s probably mostly hidden inside of some kind of application framework. However, if you wanted to use Lamar all by itself you would first bootstrap a Lamar container with all its service registrations something like this:


var container = new Container(x =>
{
    // Using StructureMap style registrations
    x.For<IClock>().Use<Clock>();
    
    // Using ASP.Net Core DI style registrations
    x.AddTransient<IClock, Clock>();
    
    // and lots more services in all likelihood
});

Now, to resolve services from your container:


// StructureMap style

// Get a required service
var clock = container.GetInstance<IClock>();

// Try to resolve a service if it's registered
var service = container.TryGetInstance<IService>();

// ASP.Net Core style
var provider = (IServiceProvider)container;

// Get a required service
var clock2 = provider.GetRequiredService<IClock>();

// Try to resolve a service if it's registered
var service2 = provider.GetService<IService>();

Definitely note that the old StructureMap style of service resolution is semantically different than ASP.Net Core’s DI resolution methods. That’s been the cause of much user aggravation over the years.

 

Brief History of Lamar

I’ve become increasingly tired and frustrated with supporting StructureMap for the past several years. I also realized that StructureMap had fallen behind many other IoC tools in performance, and I really didn’t think that was going to be easy to address without some large scale structural changes in StructureMap.

Most of my ambition in OSS over the past couple years has been on Jasper (Marten was something we needed at work at the time and I had no idea it would turn out to be as successful as it has been). I’ve been planning for years to use Roslyn’s ability to generate code on the fly as a way to make Jasper’s runtime pipeline as fast as possible without losing much flexibility and extensibility. What is now Lamar’s code generation and compilation model was originally just a subsystem of Jasper for its runtime.

Because I was so focused on Jasper’s performance and throughput, I knew I would want to move beyond StructureMap as its IoC container. I tried the built in DI container from ASP.Net Core for a little bit, but its limited feature set was just too annoying.

Hence, Lamar was born as a new IoC container project called “‘BlueMilk” that would:

  • Completely comply with ASP.Net Core’s requirements for IoC container behavior and functionality
  • Retain quite a bit of functionality from StructureMap that I wanted
  • Provide an “off ramp” for folks that depend on StructureMap today now that I’m wanting to stop support on StructureMap
  • Ditch a lot of features from StructureMap that I never use personally and cause me a great deal of heartburn supporting
  • Support Jasper’s “Special Sauce” code weaving
  • Be as fast as possible

What’s Special about Lamar?

First off, how it works is unique compared to the at least 20-30 other OSS IoC containers in .Net.  is the usage of dynamic generation of C# code at runtime and subsequent compilation via Roslyn in memory. Most other containers build and compile Expressions to Func objects in memory. You might have to take my word for this, but that’s an awful model to work with efficiently.

Other containers depend on emitting IL and that’s just not something I ever care to do again.

It’s hard to explain, but when used in conjunction with Jasper, Lamar can in many cases use inline code generation to completely replace any kind of service location calls in Jasper’s runtime handling of messages or HTTP requests.

Why would I use this over the built in DI container?

Just to be clear, Lamar is completely compliant with all of ASP.Net Core’s DI behavioral rules and expectations. Lamar even directly implements several of the DI abstractions itself this time around rather than depending on adapter libraries that kinda, sorta force your IoC container to act like what the ASP.Net Core team arbitrarily decided was the new standard for everybody.

As you may already know, ASP.Net Core comes with a simplistic DI container out of the box and many teams are frankly going to be perfectly fine with that as it is. It’s fast and has a good cold start time.

All the same, I tried to live with the built in container as it was and got way too annoyed with all the features I missed from StructureMap, and hence, Lamar was born.

Lamar has a much richer feature set that I think absolutely has some significant value for productivity over the built in container including, but not limited to:

And for those of you hyperventilating because “oh my gosh, that sounds like some magic and [conference speaker/MVP/celebrity programmer] says that all code must be painfully explicit or the world is going to end!”, you’ll definitely want to check out all of Lamar’s diagnostic facilities that help you unravel problems and understand what’s going on within Lamar.

StructureMap to Lamar

Lamar supports a subset of StructureMap’s functionality and API. There are some behavioral differences to be aware of though:

  • The lifetime support is simplified and reflects the ASP.Net Core semantics rather than the old StructureMap lifecycle support
  • Registry is now ServiceRegistry
  • You can use a mix of StructureMap-style registration and ASP.Net Core style registrations natively
  • There is no semantic difference between Add() and Use() like there was in StructureMap. Last one in wins. Just like ASP.Net Core expects.
  • Lamar supports nested containers like most .Net frameworks expect, but child containers or profiles will probably not be supported
  • Lamar doesn’t yet support setter injection (if you want it, just be prepared for an “I take pull requests” response)
  • The Container.With().GetInstance() functionality is never going to be supported in Lamar. I’m theorizing that a formal auto-factory feature later will be a better alternative anyway
  • Lamar only supports decorators for interception so far

 

Other Information:

Lamar was originally called “BlueMilk”

 

 

 

Planning the Next Couple Marten Releases

I think we’ve got things lined up for a Marten 2.9 release in the next week with these highlights:

  • Fixes for netcoreapp2.1. A user found some issues with the Linq support so here we go.
  • The upgrade to Npgsql 4.0
  • Eliminating Netstandard 1.3 support. Marten will continue to target .Net 4.6 & Netstandard 2.0, and we’ll be running tests for net46, netcoreapp2.0, and netcoreapp2.1.
  • Some improvements on exception messages in the event store support
  • Upgrades to Newtonsoft.Json — but if the performance isn’t better than the baseline, I’ll leave it where it is

Brainstorming What’s Next

I’ve been admittedly putting  most of my OSS energy into other projects (Jasper/Lamar) for the past year, but the Marten community has been happily proceeding anyway. One of my goals for OSS work this year is to get the Marten issue list count (bugs, questions, and feature ideas) under 25 and keep it there so it’s only a single page in the GitHub website.

We’ve got a big backlog of issues and feature requests. For a potential Marten 2.9, I’d like to suggest:

That’s all for the document store, so now let’s talk about the event sourcing. I don’t have a huge amount of ideas because I don’t get to use it on real projects (yet), but here’s what I think just from responding to user problems:

  • The async daemon is up for some love, and I think I’ve learned some things from Jasper work that will be beneficial back in Marten. First, I think we can finally get a model where Marten itself can handle distributing the ownership of the projections between multiple application nodes so you can more easily deploy the async daemon. After a tip from one of my new colleagues at Calavista, I also want to pursue using “tombstone” events as placeholders on event capture failures to make the async daemon potentially quite a bit more efficient at runtime.
  • I think this would be an add on, but I have an idea for an alternative to ViewProjection that would let you write your projection as just methods on a class that take in any combination of the actual event, the IDocumentSession, the containing EventStream, or Event<T> metadata, then use Lamar to do its thing to generate the actual adapter to Marten’s projection model. I think this could end up being more efficient than ViewProjection is now at runtime, and certainly lead to cleaner code by eliminating the nested lambdas.
  • Event Store partitioning?
  • Event Store metadata extensions?

EDIT 6/8/2018: We had some discussions about this list on Gitter and I definitely forgot some big things here:

  • The async daemon and projections in general needs to be multi-tenancy friendly
  • Other users are interested in the multi-tenancy via database per tenant that got cut from 2.0 at the last minute

Any thoughts about any of this? Other requests? Comments are open, or head to Gitter to join the conversation about Marten.

 

Marten 3.0 with Sql Server Support????

I want to hold the line that there won’t be any kind of huge Marten 3.0 until there’s enough improvements to the JSON support in Sql Server vNext to justify an effort to finally make Marten database engine neutral. I’m also hoping that Marten 3.0 could completely switch all the Linq support to using queries based on SQL/Json too. See this blog post for why Marten does not yet support Sql Server.

Compiling Code At Runtime with Lamar – Part 2

In my previous post, I introduced Lamar‘s ability and utilities for generating code in memory and compiling that code into usable, in memory assemblies with the Roslyn compiler. In this post I’m going to try to explain the more complicated “frames” model with Lamar that provides the backbone for the related Jasper framework’s middleware and runtime pipeline strategy.

The purpose of the “frames” model is to be able to generate dynamic methods by declaring a list of logical operations in generated code via Frame objects, then let Lamar handle:

  • Finding any missing dependencies of those frames
  • Determine what the necessary variable inputs are
  • Ordering all the frames based on dependency order just prior to generating the code

Before diving into an example, here’s a little class diagram of the main types:

LamarCodeGenClassDiagram

The various types represent:

  • Frame – Named after the StackFrame objects within stack traces in .Net. Models a logical action done in the generated code. Concrete examples in Lamar or Jasper would be calling a method on an object, calling a constructor function, or specific frame objects to create wrapped transaction boundaries or exception handling boundaries.
  • Variable – pretty well what it sounds like. This type models a variable within the generated method, but also includes information about what Frame created it to help order the dependencies
  • IVariableSource – mechanism to “find” or create variables. Examples in Lamar include resolving a service from an IoC container, passing along a method argument, or the example below that tells you the current time
  • IMethodVariables – interface that is used by Frame classes to go find their necessary Variable dependencies.

Alrighty then, let’s make this concrete. Let’s say that we want to generate and use dynamic instances of this interface:

public interface ISaySomething
{
    void Speak();
}

Moreover, I want a version of ISaySomething that will call the following method and write the current time to the console:

public static class NowSpeaker
{
    public static void Speak(DateTime now)
    {
        Console.WriteLine(now.ToString("o"));
    }
}

Our dynamic class for ISaySomething will need to pass the current time to the now parameter of that method. To help out here, there’s some built in helpers in Lamar specifically to write in the right code to get the current time to a variable of DateTime or DateTimeOffset that is named “now.”

To skip ahead a little bit, let’s generate a new class and object with the following code:

// Configures the code generation rules
// and policies
var rules = new GenerationRules("GeneratedNamespace");

// Adds the "now : DateTime" variable rule to 
// our generated code
rules.Sources.Add(new NowTimeVariableSource());

// Start the definition for a new generated assembly
var assembly = new GeneratedAssembly(rules);

// Add a new generated type called "WhatTimeIsIt" that will
// implement the 
var type = assembly.AddType("WhatTimeIsIt", typeof(ISaySomething));

// Getting the definition for the method named "Speak"
var method = type.MethodFor(nameof(ISaySomething.Speak));

// Adding a frame that calls the NowSpeaker.Speak() method and
// adding it to the generated method
var @call = new MethodCall(typeof(NowSpeaker), nameof(NowSpeaker.Speak));
method.Frames.Add(@call);

// Compile the new code!
assembly.CompileAll();

After all that, if we interrogate the source code for the generated type above (type.SourceCode), we’d see this ugly generated code:

    public class WhatTimeIsIt : Lamar.Testing.Samples.ISaySomething
    {


        public void Speak()
        {
            var now = System.DateTime.UtcNow;
            Lamar.Testing.Samples.NowSpeaker.Speak(now);
        }

    }

Some notes about the generated code:

  • Lamar was able to stick in some additional code to pass the current time into a new variable, and call the Speak(DateTime now) method with that value.
  • Lamar is smart enough to put that code before the call to the method (that kind of matters here)
  • The generated code uses full type names in almost all cases to avoid type collisions rather than trying to get smart with using statements in the generated code

So now let’s look at how Lamar was able to add the code to pass along DateTime.UtcNow. First off, let’s look at the code that just writes out the date variable:

public class NowFetchFrame : SyncFrame
{
    public NowFetchFrame(Type variableType)
    {
        // there's some sleight of hand here that's marking
        // this new Variable as created by this frame object
        // so that the dependency relationship is made
        Variable = new Variable(variableType, "now", this);
    }
    
    public Variable Variable { get; }
    
    public override void GenerateCode(
        GeneratedMethod method, 
        ISourceWriter writer)
    {
        writer.WriteLine($"var {Variable.Usage} 
            = {Variable.VariableType.FullName}.{nameof(DateTime.UtcNow)};");
        Next?.GenerateCode(method, writer);
    }
}

In the frame above, you’ll see that the GenerateCode() method writes its code into the source, then immediately turns around and tells the next Frame – if there is one – to generated its code. As the last step to write out the new source code, Lamar:

  1. Goes through an effort to find any missing frames and variables
  2. Sorts them with a topological sort (what frames depend on what other frames or variables, what variables are used or created by what frames)
  3. Organizes the frames into a single linked list
  4. Calls GenerateCode() on the first frame

In the generated method up above, the call to NowSpeaker.Speak(now) depends on the now variable which is in turn created by the NowFetchFrame, and that’s enough information for Lamar to order things and generate the final code.

Lastly, we had to use a custom IVariableSource to teach Lamar how to resolve the now variable. That code looks like this:

public class NowTimeVariableSource : IVariableSource
{
    public bool Matches(Type type)
    {
        return type == typeof(DateTime) || type == typeof(DateTimeOffset);
    }

    public Variable Create(Type type)
    {
        if (type == typeof(DateTime))
        {
            return new NowFetchFrame(typeof(DateTime)).Variable;
        }

        if (type == typeof(DateTimeOffset))
        {
            return new NowFetchFrame(typeof(DateTimeOffset)).Variable;
        }

        throw new ArgumentOutOfRangeException(nameof(type), "Only DateTime and DateTimeOffset are supported");
    }
}

Out of the box, the Lamar + Jasper combination uses variable sources for:

  • Services from the internal IoC container of the application
  • Method arguments
  • Variables that can be derived from a method argument like HttpContext.Request
  • The “now” convention shown here

Summary

I don’t know how popular this thing is going to be, but it’s powering the dynamic code generation of Jasper’s runtime pipeline and it’s the key to Jasper’s efficiency compared to other .Net tools with similar functionality. The early feedback I got was that this model was very abstract and hard to follow. I’m open to suggestions, but the very nature of needing to do the recursive dependency detection and ordering kind of necessitates a model like this in my opinion.

Compiling Code At Runtime with Lamar – Part 1

This code was originally written and proven out in the related Marten and described in a post titled Using Roslyn for Runtime Code Generation in Marten. This code was ripped out of Marten itself, but it’s happily running now in Lamar a couple years later.

As some of you know, I’ve been working on a new library called Lamar that I mean to be the next generation replacement for the venerable, but creaky StructureMap library. Besides the IoC Container support though, Lamar also provides some tooling and a model to generate and compile code at runtime, then ultimately load and use the newly generated types.

If all you want to do is take some C# code and compile that in memory to a new, in memory assembly, you can use
the Lamar.Compilation.AssemblyGenerator class.

Let’s say that you have a simple interface in your system like this:

    public interface IOperation
    {
        int Calculate(int one, int two);
    }

Next, let’s use AssemblyGenerator to compile code with a custom implementation of IOperation that we’ve generated
in code:

        [Fact]
        public void generate_code_on_the_fly()
        {
            var generator = new AssemblyGenerator();

            // This is necessary for the compilation to succeed
            // It's exactly the equivalent of adding references
            // to your project
            generator.ReferenceAssembly(typeof(Console).Assembly);
            generator.ReferenceAssembly(typeof(IOperation).Assembly);

            // Compile and generate a new .Net Assembly object
            // in memory
            var assembly = generator.Generate(@"
using Lamar.Testing.Samples;

namespace Generated
{
    public class AddOperator : IOperation
    {
        public int Calculate(int one, int two)
        {
            return one + two;
        }
    }
}
");

            // Find the new type we generated up above
            var type = assembly.GetExportedTypes().Single();
            
            // Use Activator.CreateInstance() to build an object
            // instance of our new class, and cast it to the 
            // IOperation interface
            var operation = (IOperation)Activator.CreateInstance(type);

            // Use our new type
            var result = operation.Calculate(1, 2);
         
            result.ShouldBe(3);
        }

There’s only a couple things going on in the code above:

  1. I added an assembly reference for the .Net assembly that holds the IOperation interface
  2. I passed a string to the ​GenerateCode() method, which successfully compiles my code and hands me back a .Net Assembly object
  3. Load the newly generated type from the new Assembly
  4. Use the new IOperation

If you’re not perfectly keen on doing brute force string manipulation to generate your code, you can
also use Lamar’s built in ISourceWriter to generate some of the code for you with
all its code generation utilities:

[Fact]
public void generate_code_on_the_fly_using_source_writer()
{
    var generator = new AssemblyGenerator();

    // This is necessary for the compilation to succeed
    // It's exactly the equivalent of adding references
    // to your project
    generator.ReferenceAssembly(typeof(Console).Assembly);
    generator.ReferenceAssembly(typeof(IOperation).Assembly);


    var assembly = generator.Generate(x =>
    {
        x.Namespace("Generated");
        x.StartClass("AddOperator", typeof(IOperation));
        
        x.Write("BLOCK:public int Calculate(int one, int two)");
        x.Write("return one + two;");
        x.FinishBlock();  // Finish the method
        
        x.FinishBlock();  // Finish the class
        x.FinishBlock();  // Finish the namespace
    });

    var type = assembly.GetExportedTypes().Single();
    var operation = (IOperation)Activator.CreateInstance(type);

    var result = operation.Calculate(1, 2);

    result.ShouldBe(3);
}

In Part 2, I’ll talk about the “frames” model that’s heavily used within Jasper (shown in this post).