General

Multi-tenancy in ASP.NET MVC - Controller Actions (Part I)

Rob Ashton — Sun, 14 Feb 2010 17:17:29 GMT

Previous entries in the series

In the last entry, we covered how and why we might want to replace existing views and partial views from the core application with our own from modules. We also covered that with this ability it was possible to add entirely new views and partial views.

However, views need actions and actions come from controllers. If we add a new view to the application and the core application does not support that path with an action, that view cannot be loaded.

It stands to reason therefore that our modules need the ability to add new controller actions (and indeed replace existing actions) at run-time on a per-request basis - again assuming we're going for full on multi-tenancy.

Actions come from controllers, and by default controllers come from the main web assembly. Now obviously our modules should be as self contained as possible and therefore probably each have their own assemblies so they can be developed separately and added to the project ad-hoc.

Once again, the ASP.NET MVC team have given us an extensibility point with which to override this default behaviour with the ability to implement our own controller factories.

Resolving actions
As with the last topic, I will assume the presence of a configuration provider that can tell us which modules are loaded.
For purposes of simplicity, the Module class now contains a reference to an Assembly that we'll assume was loaded in when the configuration was last scanned.

    public class Module
    {
        public string Id
        {
            get;
            set;
        }

        public Assembly Assembly
        {
            get;
            set;
        }
    }

The job of the ControllerFactory is another well documented concept; when a controller is required, the factory is invoked with the name of the controller being requested and the current request data. It is expected to return an instance of the controller (which is used for that single request), and then just like the ViewEngine is given that controller to dispose of at the end of the request.

The simplest solution is clearly going to be that we look at the context we have access to and then work out which controller to return based on that context.
The context in this case being the name of the controller, the action being requested and the collection of modules which are currently active for this request.

Each module can therefore hold their own controllers with their own actions, and the controller factory can select which controller to return when a specific action is being invoked.

I implement my controller factory from the base interface, which is System.Web.Mvc.IControllerFactory:

    public class ModuleControllerFactory : IControllerFactory
    {
        public IController CreateController(System.Web.Routing.RequestContext requestContext, string controllerName)
        {
            throw new NotImplementedException();
        }

        public void ReleaseController(IController controller)
        {
            throw new NotImplementedException();
        }
    }

ReleaseController can just check for IDisposable and dispose if necessary, so we'll take that as read and focus on what we need to do in order to create the controller.

The Goal

These are the modules exposed by the application, the controllers they provide and the actions those controllers have on them.

CoreModule
- HomeController
- ActionResult Index();
ModuleOne
- HomeController
  - ActionResult Extra();
ModuleTwo
- HomeController
  - ActionResult Index();
  - ActionResult Extra();
- OtherController
  - ActionResult Index();

WIth the set-up, the following behaviour is desired:

Core Module loaded:

/Home/Index requested => CoreModule Index Invoked
/Home/Extra requested => Action Not found

Core + ModuleOne Loaded

/Home/Index requested => CoreModule Index Invoked
/Home/Extra requested => ModuleOne Extra Invoked

Core + ModuleOne + ModuleTwo Loaded (in that order)

/Home/Index requested => ModuleTwo Index Invoked
/Home/Extra requested => ModuleTwo Extra Invoked
/Other/Index requested => ModuleTwo Index Invoked

A Solution

This is actually quite tricky, as the solution is going to involve not only scanning for the controllers, but scanning for methods on those controllers that match the actions being requested.

There are a lot of rules involved already in MVC selecting the right method to call from a controller, and we don't want to go down the route of duplicating this, so this is where we set a convention and say that if *any* action is found with the name being requested, that we'll use that controller and assume that all the necessary permutations of that action will be provided too. (A post action vs Get action for example).

There are two parts to solving this problem, finding the type we want to create, and creating the controller from that type.

The process will be similar to that of the ViewEngine example:

1) Reverse the module list order so we have the most recently loaded first
2) Scan all the types in the assembly for that module
3) Find a type with the name we're looking for ( <Name>Controller )
4) Scan the methods on that type to find the action we're looking for
5) If found, return this type
6) Else Continue

Obviously reflecting on all these types is a slow process, and we should cache the type once found by configuration id, controller name and action name.

Here is some code which loosely achieves the above:

private Type FindControllerType(String controllerName, RequestContext requestContext, Configuration currentConfiguration)
        {
            // Generate the type name we're looking for
            String controllerTypeName = string.Format("{0}Controller", controllerName);

            // Get the action and therefore method name we're looking for
            String actionName = (string)requestContext.RouteData.Values["action"];

            // TODO: Check Cache here

            // Get modules in reverse order
            var searchModules = currentConfiguration
                .Modules
                .Reverse();

            foreach (var module in searchModules)
            {
                // Get all the types in the assembly
                Type[] controllerTypes = module.Assembly.GetTypes()
                    .Where(
                        t =>
                            // Where the type name is the one we're looking for
                            t.Name == controllerTypeName &&

                            // Where it can be cast to a controller
                            typeof(IController).IsAssignableFrom(t) &&

                            // And there is a public instance method with the name we're looking for on that type
                            t.GetMethods(System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Instance)
                            .Where(m => m.Name == actionName).Count() > 0

                        ).ToArray();

                // Skip to the next module if no types found
                if (controllerTypes.Length == 0) { continue; }

                // Else, simply return the first one found
                return controllerTypes[0];
            }

            // Fail
            return null;
        }

A very rudimentary controller factory implementation would therefore look something like this:

 public class ModuleControllerFactory : IControllerFactory
    {
        private IConfigurationProvider mConfigurationProvider;

        public ModuleControllerFactory(IConfigurationProvider configurationProvider)
        {
            mConfigurationProvider = configurationProvider;
        }

        public IController CreateController(System.Web.Routing.RequestContext requestContext, string controllerName)
        {
            Type t = FindControllerType(controllerName, requestContext, mConfigurationProvider.GetActiveConfiguration());
            return (IController)Activator.CreateInstance(t);
        }

        public void ReleaseController(IController controller)
        {
            IDisposable disposable = controller as IDisposable;
            if (disposable != null)
            {
                disposable.Dispose();
            }
        }
// Etc

It is of course probably desirable to instantiate the controller using your favourite IOC container - so for StructureMap for example instead of using Activator.CreateInstance you would call ObjectFactory.GetInstance(t);

(Actually, you'd probably inject the container as well rather than calling ObjectFactory directly but you get the gist).

Summary

Essentially, we can completely re-wire this part of the ASP.NET MVC framework to do what we want it to do. We can load our controllers from wherever based on whatever context we like - and this gives us a powerful mechanism for pluggability and therefore multi-tenancy.

I did contemplate trying to achieve this through routing - custom routing constraints and handlers - but it's not a tidy solution, it generally means having different names for your controllers, or playing havok with namespaces and configuration and because a lot of that configuration is static it often involves re-compilation.

Re-compilation is something to be avoided, as we ideally want to be able to add new customers by just modifying configuration.

Anyway, there are a number of options and this is just one of them, I'll be hoping to cover a crazy solution using Reflection.Emit and hopefully delve into MEF before I'm done with this particular part of the multi-tenancy story.

DDD8 Feedback... feedback

Rob Ashton — Tue, 09 Feb 2010 17:28:36 GMT

The feedback from DDD8 has come in, in anonymous form and for the most part I'm completely overwhelmed by the positive response the talk garnered from the people who attended the talk.

However, I get the feeling my talk managed to polarise the room somewhat and I'd like to respond to those that went completely the opposite way with their responses and apologise if I went a different direction to what they expected!

Varied Demos
Okay, no defence here at all - showing you all the same website over and over again until I got around to showing the final product of the concepts( NerdDinner) was a bit unfair - I'll try to put together the demos a bit earlier than 3am the night before next time :)

I think perhaps concentrating on NerdDinner from the start and showing how I built that up might have helped, but when I put the demos together I was struggling to simplify that enough for the demos to make sense in their own right (hence the hurried individual ones)

Not enough Detail
I never intended on going into detail on the concepts outlined - I don't feel you ever can go into detail in a talk that lasts less than an hour - nevertheless, if you'd like more detail you're all more than welcome to respond to my series on multi-tenancy with questions and more feedback!

I already have a longer slot booked at one user group - and would be happy to spend more time on the subject at any other user groups or developer meet-ups - just ask.

Too technical
I should have made clear in my summary that a prior knowledge of ASP.NET MVC would be useful - and I apologise for this - a few people complained about this and I'll know next time to be more up front about what I intend on covering in a session.

Too fast
This is probably one of the most subjective responses I had, and a lot of people said they loved the pace - which kept them awake and interested throughout the talk. I don't think I'll change this in my future talks (although I might try to talk slower), but another disclaimer may well not go amiss "This talk will be fast-paced and try to throw as much information at you in an hour as your brain can cope with"

Wrong approach
Okay, only one person mentioned this - and said I should be using IOC to solve the issues I outlined. I'm not entirely sure (apart from the controllers) where this would have helped - so feel free to contribute to the multi-tenancy conversation in the comments of this blog when I get to that point whoever you are!

I was also criticised by the same person for even wanting to host the same application under the same IIS instance - I believe I covered this as an option in one of my slides however and I was certainly not suggesting that I recommended keeping your customers in one basket.

I think there are masses of conversations and discussions to be had over the approach I took to solving the problem - which is why I'm now doing this blog series - to get the discussion going and come to an agreed solution between us all.

Poor distribution of Swag
Yes yes yes, I'm sorry - I'm really bad at this kind of thing because it's not something I'm interested in - I'll try to do better next time.

Too nervous
Guilty as charged, my first time speaking in a while - I'll try to be more confident next time

Too confident/arrogant
Towards the end perhaps you're right, I'm trying to work out whether this is better or worse than the above!

Ending on a high note
In response to all the people who said my talk had great pace, great content, that it was incredibly useful and relevant, that the subject area had justice given to it, and that I clearly knew what I was talking about - thanks a bunch, rest assured that the above comments have balanced out the ego and I can still fit through doors.

Thanks to everybody who attended and asked questions - I'll be seeing you all again shortly no doubt.

Multi-tenancy in ASP.NET MVC - Why do we want it?

Rob Ashton — Mon, 01 Feb 2010 22:10:22 GMT

I'll be gratuitously "borrowing" a lot of material from my DDD8 slides in this post, it seemed like the right thing to do given that this series is a write-up and then continuation of that talk.

When dealing with more than one customer in the desktop market, it is customary to have a single product which is extendable through the use of plug-ins and an API, and often you can leave it up to your consumer base to write those plug-ins and add to your product in a manner they see fit.

In the web world it's a bit different, and you don't typically get that kind of behaviour (Facebook applications may or may not count, depending on how you look at it).

In a simple world, you'll have a single product which is used directly off the shelf by multiple customers:

When building web applications for a varied and paying customer base, It is likely that you have customers that are fickle and will want things done their way.

It often does not make business sense to turn good business down, and the business is what pays the hungry developer and thus when you finally get a customer who wants things done differently, the business tells the developer to jump and the business's required response is the proverbial "how high?".

Consider the above diagram then, and imagine Customer A asking for something 'just a little bit different' and think about what your options could be.

We'll get the obvious dusted out of the way first:

When I switched to the above code as a slide in my multi-tenant talk, I was greeted with laughter, but we've all known products which have ended up with such delightful nuggets in them.
It's blindingly obvious that it's not the right solution, and that as you progress down the route of making further modifications for either Customer A, B or C you'll end up with an un-maintainable mess of switches and flags. 'Nuff said.

You could then decide that you're going to keep that customer as a new product in its own right - that would remove the need for all those on-off switches.

This should very obviously be a big no-no as maintaining that then bespoke product and keeping it up to date with any changes then made to your core product is going to be nothing more than a giant headache.
As you get more customers, the number of developers you'll need to hire will increase almost in direct proportion to the number of codebases you have to maintain! (Again, I've seen this done - so don't think I'm just pointing out the obvious for the sake of it)

Let's move onto the more-often used approach, of branching from a base product for your different needs, and utilising the power of a source control system to keep changes in sync between your code-bases.

Customer A can be kept up to date by merging changes from the core product, and Customer B/C can get additional features from Customer A's branch if and when they desire it.

At first glances, it seems like this solution fits our needs - and indeed it can work well in a lot of given scenarios. The problem comes when you scale this solution up to more than this small example - as few of us are (un)lucky enough to be able to deal with only three customers and remain financially viable!

Here is a small example of 30 customers sharing 15 code-bases!

Yowsers!
How do you keep track of who has what features?
How do you test all of those different branches of code?
How do you deploy those branches of code?
How do you make a new version of the product and serve it to a customer?
How many developers do you need to manage that process?

It's never as simple as it looks, and you end up with not only the above problems, but you end up with the additional problems of what happens when a branch becomes radically different and you're unable to merge changes around.
There is too much developer interaction here - and your skilled staff end up having to spend most of their time creating new branches/pushing changes around instead of spending their time doing what they're actually trained to do - writing code.

Adding a new customer shouldn't be about changing code, it should be about manipulating configuration, and modifying a customer shouldn't be about changing code, it should also be about manipulating configuration.

In other words, problems should only be solved once - and configuration should be used to give or take features to and from customers.

Enter multi-tenancy...

The core concept of a well written multi-tenant application is that you should have a single code base, and a number of configurations - where each configuration tells the runtime what functionality should be available and what the look/feel should be.

Before continuing, I'd like to define a few of the terms I'll be using throughout this series of blog entries.

Module: A discrete set of functionality
Theme: The look and feel
Configuration: A selection of modules, and a single theme

This is a personal leaning, and I know that some people would set this up differently. Each to their own, we've got to draw lines somewhere!

Anyway - as far as I'm concerned, Multi-tenancy gives us some of the following benefits:

Deployment becomes a simple case of installing your application onto a server, and setting up the configurations for that application.

When a request comes in, context is determined by some means (auth credentials, the hostname, whatever), and the relevant configuration is selected from that context.

This is a very simple way of working, and if you design your application correctly, it becomes obvious that your hosting/maintenance costs can be reduced.

You can have multiple servers with the exact same codebase installed on them, and with all the configurations available to them (In other words, identical). Scaling up becomes a simple matter of adding more of those identical servers - and if you're really smart you can load balance across your VPSs and power them up/down as required.
You no longer need to worry (too much) about the fact that you have all of those customers, and you can concentrate on the health of your system as a whole.

Some more benefits:

You add a feature once, and deploy it to your customers through the use of configuration
You can fix a bug, and deploy the fix once
Potentially easy management of your infrastructure (This actually comes through good design, and multi-tenancy just aids in that goal)
Developers get to spend their time coding new features/fixing bugs
New customers can have a site created in minutes and start to give feedback immediately

Everybody is a winner and we all get to go home and have pie and punch.

In the next entry, things will hot up as I'll start to look at ASP.NET MVC and determine the components that we can use to aid us in creating a multi-tenant application.

Multi-tenancy and ASP.NET MVC

Rob Ashton — Sun, 31 Jan 2010 20:16:24 GMT

I spoke about this chestnut briefly at DDD8, and I want to start expanding on the subject.

My plan is over the next few weeks to start talking more about multi-tenancy in our web-apps, and to get everybody else doing the same - speaking to other developers after my talk I realised that we're not alone, people are working on solutions but they're just not talking about it.

By getting some dialogue going, I hope we can generate a public description of what is good and what is bad about attempting to build multi-tenant applications on top of ASP.NET MVC, and what our possible avenues of achieving this can be.

The suggested topics I aim to cover in this series of blog entries will (to begin with) be somewhere along these lines: (As I've already written most of the material!)

What is multi-tenancy and why do we want it?
The building blocks of a multi-tenant application in ASP.NET MVC

How I integrated MvcEx into NerdDinner to give it some multi-tenant capabilities

If there is anything else you think I should cover as part of this series, then let me know by either Twitter or the comments field below.

I am loathe to go into detail on what I consider to be the other 90% of the multi-tenancy story - your domain, the rest of your codebase, managing your databases/configurations etc, as you can get all of that information from people who are far more versed in the subject than I.

But, if pushed on a particular subject I guess I will describe how I deal with those issues in the codebases I have control over, and you'll have to take it (as you should take anything written on these pages) with a pinch of salt.

DDD8 - Post Event

Rob Ashton — Sun, 31 Jan 2010 18:01:55 GMT

Wow.

That was a wonderful day, and the sessions I ended up going to were:

@ICooper's session on MVC Architecture (preaching the choir but good to be re-assured)
@robashton's session on Multi-tenant ASP.NET MVC (obviously)
@holytshirt's session on Mono (Good to see this project is advancing well)
@garyshort's session on JClosure (Lovely!)
@blowdart's session on the crystal maze

The last session was interrupted constantly by the MVPs and associated crowd because Barry is leaving the UK and heading off to MS to learn how to spell :)

There isn't much specifically to talk about really, it's been said by everybody. The event was well organised, the post-event meal was also surprisingly kept in control and the post-event drinks ... well I'd had no sleep the previous night so I left early. I believe fun was had by all however.

My talk? I think it went okay - I was a bit nervous presenting on a subject that doesn't get talked about openly all that much, and worried the audience might throw a few massive spanners in the works (although I am open to change, I don't want to be told outright I'm wrong in the middle of a talk!)

I spoke a bit fast, and had a minor emergency at the start when realising I needed an adapter for my laptop, but was saved by the team whose job it was to babysit me and massive thanks goes to them for saving my presentation :)

My slides can be found here, and demo code can be found here.

The nerd dinner multi-tenant example can be found on the MvcEx codeplex site (http://mvcex.codeplex.com) - but I'll be hoping to improve it beyond its "suitable for demo" stage and do some blog entries on the rationale behind some of the decisions/concepts found within over the coming weeks.

It's not perfect, it's not anywhere near done and as I keep telling people, it's just for reference purposes (at present), feel free to make suggestions, contributions and etc and we'll get there in the end. Multi-tenancy is the future you know?

NUnit 2.5.3 and [SetUICulture]

Rob Ashton — Fri, 18 Dec 2009 14:28:11 GMT

I can't for the life of me get this to work, and [SetCulture] appears to be working fine - so I can only assume it's a bug.

I've posted to the mailing list and started off the process of working out whether it is a bug or not, but for now - I need to have my tests running in the right culture, without any side effects on the other tests once a test has been complete.

Here is my hack to do that:

   public class CultureContext : IDisposable
    {
        private CultureInfo mOldCulture;

        public CultureContext(String cultureName)
        {
            mOldCulture = System.Threading.Thread.CurrentThread.CurrentUICulture;
            System.Threading.Thread.CurrentThread.CurrentUICulture = new CultureInfo(cultureName);
        }

        public void Dispose()
        {
            System.Threading.Thread.CurrentThread.CurrentUICulture = mOldCulture;
        }
    }

The usage is as follows, within a test do:

using (new CultureContext("fr-FR"))
{
	// Test code here
}

This will ensure that your test runs with the ui culture of "fr-FR", before resetting it to whatever it was before the test began. Not pretty, but it'll do until I work out if it's user error or a bug preventing NUnit from doing what I want it to do!

DDD8

Rob Ashton — Fri, 11 Dec 2009 11:56:39 GMT

Developer Developer Developer Day 8 has been announced, and I'm going to propose a couple of sessions, and hopefully use one of them to talk about an open source project I've been cooking for a month or so now :)

Click here for details!

Legacy .NET 1.1 COM in a classic ASP application after installing the .NET 2.0 Runtime

Rob Ashton — Wed, 28 Oct 2009 23:32:47 GMT

I've just had a fun evening trying to come up with a half-suitable solution for this problem and thought it worth documenting here although my next entry isn't due for a couple of days.

As previously mentioned, we run a legacy classic ASP system for most of our clients and we're in the process of writing a replacement system in .NET.

This week we finished the user acceptance testing on the first deployable .NET application - a small interface built as a replacement for a small portion of that classic ASP system and needed therefore to build it to a live server.

The servers are fairly locked down as you'd expect and a change request had to be submitted to get .NET Framework 3.5 installed along with the .NET 2.0 runtime. I installed some routine security updates at the same time and rebooted the servers one at a time whilst testing that the legacy sites still worked.

All seemed great until the next day when a very tired CTO turned up asking if he could get some sleep tonight because he had been receiving Nagios alerts all night telling him that the live sites were failing over.
Looking at the logs we could see that the application pools for most of the legacy sites had been recycling every hour and freezing for a vast majority of that hour - and with a bit of creative debugging the problem was isolated to the CreateObject calls to a pile of both in-house and third party managed COM objects installed on the server.

Rather than failing gracefully and logging an error, for some reason the calls were simply backing the server up and causing it to stop responding to requests.

I had a hunch that this was to do with the new version of the runtime we had installed and set about trying to prove that this was the case.

When an un-managed process (in this case our IIS worker process) is started up, it is obviously done so without initializing the .NET framework. If you then attempt to load a managed assembly into that process (in this case our COM object), the most recent version of the framework will be loaded into that process.

At the moment, only one version of the .NET Framework is capable of being loaded into a single process, and this is determined on a first come first served basis. 99% of the time if you load a .NET 1.1 assembly into a .NET 2.0 process, things will work fine because there are few breaking changes between the two runtimes - however there are cases where this does not hold true and as people always say "your mileage may vary".

In IIS, the application was set up as being a .NET 1.1 website - but I wasn't convinced that this had much to do with anything and decided to try out a little hack based on my previous experience with desktop application development, COM and the different .NET runtimes.

When you deploy a .NET application, you can supply a configuration file with various overrides/version-mappings/etc alongside that application with the convention <applicationname>.exe.config. This means that you can also supply a configuration file for an application you don't "own" and modify that way it works on your machine.

<?xml version="1.0"?>
<configuration>
 <startup>
 <supportedRuntime version="v1.1.4322"/>
 <requiredRuntime version="v1.1.4322"/>
 </startup>
</configuration>

Some of the suggestions on the internet were to stick this in a web.config file at application root, but:

I've read elsewhere that this is not supported in web applications
I had a feeling that this directive would be ignored because the web.config file would only be read when a .NET resource was requested.

I decided that my nail was much bigger, and instead created an inetsrv.exe.config, a w3wp.exe.config and a dllhost.exe.config with the above directive in and placed them in system32/inetsrv and system32 respectively.

Restarting IIS and hitting my test page with all the CreateObjects (my fail case) yielded in a success and my next step was therefore going to be to find something that didn't involve forcing all IIS processes and god knows what else to run with .NET 1.1 and .NET 1.1 only.

As previously mentioned, which runtime loaded into the worker process depends entirely on who first requests that a .NET assembly be loaded into process. The alternative in the desktop world and certain other places would be to manually invoke the .NET runtime with an explicit call to CorBindToCurrentRuntime or similar.

I decided the best approach would be to force the IIS worker process to do this for me by invoking some .NET code in the context of the application. This would then use the framework version specified in the application and solve all the problems.

A few options were considered such as:

Creating a global.asax to get executed when application starts

Creating a web.config file as outlined in a few of the responses to questions like this on the internet

Creating a dummy aspx file which could be requested by global.asa on application startup

1 and 2 wouldn't work because no requests are ever made to the server through the aspnet extension, so 3 seemed to be the only reasonable solution.

The following aspx file was created on the server:

<%@ Page Language="vb" AutoEventWireup="false" %>

<p>Hello World</p>

And the following code added to global.asa:

if not Application("hasInitializedNet") then

	Dim reqUrl 
	reqUrl = "http://" & Request.ServerVariables("LOCAL_ADDR") & "/default.aspx"

	Dim objXmlHttp
	Dim strHTML

	Set objXmlHttp = Server.CreateObject("Msxml2.ServerXMLHTTP")

	objXmlHttp.open "GET", reqUrl , False

	objXmlHttp.send

	strHTML = objXmlHttp.responseText
	
	Set objXmlHttp = nothing

	Application("hasInitializedNet") = true

end if

The application was restarted and voila, the page was requested when the first user hit the server and the right version of.NET was loaded into the application from the start.

I can't think of a neater way that would allow us to still have our .NET 3.5 applications on the same machine and this solution is good enough for now (Until we get another pair of servers in for the new systems).

Feel free to leave any better suggestions in the comments!

Technorati tags: COM, ASP.NET, .NET Runtime, ASP