Hello World 🙂

It’s been so long since I last blogged, it essentially feels to be blogging like first time. A lot has happened all this time, and I hope I would be able to share some of my learnings more frequently going forward.

Switching to the topic of the current blog post, I got a chance to do some serious development with Managed Extensibility Framework recently. And I must say, I was thrilled to see the completely new definition of “extensibility” put forth by this framework; with the degree of easiness, decentralization and dynamicity this framework brings in enabling injection of third-party components in your code.

In my case, I was developing a sort of ETL (Extract/Transform/Load) utility. It needed to read data from a database, apply various checks and/or transformations to ensure the data was in the desired format; and upon success of all checks, add the data to a second database where it would be processed further by downstream applications.

We needed to provide extensibility to clients for the checks the data went through to decide whether it should be added to the second database. We were shipping a couple of checks out-of-the-box and clients needed drop-in extensibility for adding new checks as desired. After some research, I came to the conclusion (MEF) Managed Extensibility Framework was the better of the options available for our current use-case.

As I was planning the implementation with MEF (which basically consisted of just publishing an interface that plugins can implement to perform the check, and importing all such plugins using the ImportMany attribute of MEF), there came the inevitable question of managing the dependencies between various checks. Most of the extensibility we needed to provide was already well demonstrated by the SimpleCalculator example on MEF’s MSDN page. However unlike the example, where the exports that extended the functionality of the Calculator were independent of each other, the exported checks in our case needed to be able to express dependencies between them so as to ensure they are always executed in the right order.

Out of the box, MEF ships with the assumption that parts (both Imports and Exports) are independent and can function with no knowledge of each other. However it leaves enough room for expressing dependencies and any other custom information by parts if required using Metadata.

The easiest (and in fact the first) solution that came to my mind was using a numeric Metadata property let’s say Position, so the interface definition for metadata (that MEF calls metadata views) would look like:

 

{syntaxhighlighter brush: csharp;fontsize: 100; first-line: 1; }public interface IPluginMetadata
{
[DefaultValue(1)]
int Position { get; }
}{/syntaxhighlighter}

Plugins would define their relative ordering using a numeric value for Position metadata property and any plugin that doesn’t define Position explicitly can be executed arbitararily in any order between other plugins. Plugins specifying the Position metadata property would be executed in increasing order of the value for this metadata property.

However as I thought more into this approach, I was not quite satisfied with it. It did not seem to have enough flexibility to allow precise specification of relative ordering of plugins. For example, if someone created a plugin with Position 5 as it needed to be executed after another existing plugin with Position 4, there would be no way a third plugin can be deterministically executed after plugin with Position 4 but before Position 5 plugin. In most cases, this would seem far-fetched and in our case also, we did not need such precise specification of plugin ordering; I was nevertheless interested in finding a better alternative both from an academic perspective; as well as practically too. Recall I mentioned we were shipping 2 checks out of the box for our ETL application (where a check was basically an imported MEF plugin); what if a client later wanted to inject a custom operation between the two being shipped out of the box. This could have happened down the road, and the numerical ordering approach didn’t just seem flexible enough and not something that complements the rich extensibility and abstraction offered by the MEF framework.

As I was thinking and researching more into the problem, I came across the OrderAttribute from Visual Studio’s SDK. And straight away, I very much liked the idea. Give a unique name to each plugin and allow plugins to specify their relative ordering by explicitly communicating to the Host the list of other plugins which should be executed before or after them.

The OrderAttribute referenced came from Visual Studio’s SDK, and I did not want to reference VS assemblies in our solution, so I decided to replicate the functionality inspired from VS. Thus my OrderAttribute looked like this:

 

{syntaxhighlighter brush: csharp;fontsize: 100; first-line: 1; }[MetadataAttribute]
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method | AttributeTargets.Property | AttributeTargets.Field, AllowMultiple = true)]
public class OrderAttribute : Attribute
{
#region Public Properties
public string Before { get; set; }
public string After { get; set; }
#endregion
}{/syntaxhighlighter}

 

The next challenge was the definition of the Metadata view to support the Order metadata attribute. Notice this attribute can be applied multiple times to a plugin class to express its relative dependency to other plugins. And there was no documentation on MSDN detailing how a multiple usage MEF metadata attribute translates to a property in the metadata view. Luckily I got my answer in a comment on a MEF’s documentation page at its codeplex site. And I arrived at the following conclusion:

Multiple custom export attributes can be expressed as an array corresponding to each property of the exported attribute.

Armed with this knowledge, my Metadata view interface looked like the following:

 

{syntaxhighlighter brush: csharp;fontsize: 100; first-line: 1; }public interface IPluginMetadata
{
string Name { get; }

[DefaultValue(new string[] { })]
string[] Before { get; }

[DefaultValue(new string[] { })]
string[] After { get; }
}{/syntaxhighlighter}

At this moment, I could import multiple plugins for my purposes where the plugins can precisely define their dependencies. Here’s one such sample plugin using the Order attribute to define relative dependencies:

 

{syntaxhighlighter brush: csharp;fontsize: 100; first-line: 1; }[Export(typeof(IPlugin))]
[ExportMetadata(“Name”, “Company3.Plugin”)]
[Order(Before = “Company1.Plugin”, After = “Company2.Plugin”)]
public class MyPlugin: IPlugin
{
#region IPlugin Members
public void DoSomething ()
{
Console.WriteLine(“Hello from Company 3”);
}
#endregion
}{/syntaxhighlighter}

The beauty of the approach is you are able to specify both ways: whether you want your plugin to be executed before another plugin or after another plugin. You can decorate your plugin with with Order attribute multiple times if you have multiple Before or After dependencies (Before dependency meaning you want your plugin to be executed “before” the other plugin specified, and vice versa for the After dependency). So if you want to inject your plugin before another plugin that your client is already using, you do not need to modify the other plugin’s code (that might not even be accessible to you). You can configure your plugin to be executed before the other plugin simply by specifying the Order attribute on your plugin class using the other plugin’s name as the value for Before property of the attribute.

You do not need to define both Before and After attribute while decorating your plugin with the Order attribute, you can use only one of these if that is what you need.

The final piece of the puzzle was ordering the imported plugins in the Host class (i.e. the consumer of these plugins). Surprisingly, this proved more tricky than I initially thought. It actually took more time in writing the logic to re-order imported plugins based on their dependencies specified than it took to assemble this whole order/dependency management infrastructure. The complexity stemmed from the recursive nature of dependencies. So for example, if we have 3 plugins, A, B and C. Where A has a Before dependency on B and After dependency on C, then although there is no explicit relation between plugins B and C, but there is an implicit After relation of B on C (or C has an implicit Before dependency on B). Such implicit dependencies can cascade unexpectedly, as number of plugins as a whole as well as proportion of those having dependencies increase, so it took some good thought and time to conceptualize logic to correctly reorder plugins before they are invoked.

Here’s one sample Host class that can use such plugins and ensure they are invoked in order of their dependencies (if any):

 

{syntaxhighlighter brush: csharp;fontsize: 100; first-line: 1; }public class Host
{
#region Private Members
[ImportMany]
private IEnumerable<Lazy<IPlugin, IPluginMetadata>> plugins = null;
private List<IPlugin> sortedPlugins = null;
#endregion

#region Public Methods
public void DoWork ()
{
this.ensureParts();

foreach (var plugin in this.sortedPlugins)
{
plugin.DoSomething();
}
}
#endregion

#region Private Methods
private void ensureParts ()
{
if (this.plugins != null)
{
return;
}

var catalog = new AggregateCatalog();

catalog.Catalogs.Add(new AssemblyCatalog(typeof(Host).Assembly));
catalog.Catalogs.Add(new DirectoryCatalog(Path.GetDirectoryName(typeof(Host).Assembly.Location)));

var container = new CompositionContainer(catalog);

container.ComposeParts(this);

this.sortedPlugins = this.getSortedParts(this.plugins);
}

private List<IPlugin> getSortedParts (IEnumerable<Lazy<IPlugin, IPluginMetadata>> list)
{
//The code for sorting is available in the sample code attached with this blog post.
}
}
{/syntaxhighlighter}

Due to the length of logic that orders the plugins before consuming them, I have not re-produced it inline above. However the same is attached with the blog post below.

The attached sample solution contains 4 projects:

1. The ConsoleApplication1 project is the Host project that exposes the interface (and associated metadata view) that plugins should implement. It imports those plugins, re-orders them based on the metadata and invokes the plugins in order of their dependencies.

   The Host class in this project contains the logic for re-ordering plugins that I did not reproduce above.

2. The other 3 library projects are very skeletal implementations of the plugin interface primarily demonstrating how to specify the metadata and dependencies between them.

Please ensure to build the entire solution before trying to run the console application. There are some more points/assumptions you should take note of for this implementation of plugin re-ordering:

• Each plugin is assumed to have a unique name. This is very easy to achieve and in most cases, you can use the fully qualified name of your plugin class (i.e. its full namespace and the class name in regular dot separated notation) as the plugin metadata name.
• Plugins define their Before and/or After dependencies using the unique name of other plugins and names are case-sensitive.
• Ensure there are no circular dependencies between plugins (plugin A specifying B as its Before dependency and B specifying A as its Before dependency creates a circular dependency). I haven’t tested the code for this scenario.
• If you choose to specify only one of the Before or After properties for Order attribute on your plugin class, the other property for the Order attribute would have null value. This does not affect you if you are the plugin author.

  However if your are the plugin consumer (i.e. the author of Host class), you need to be aware that the Before and After arrays obtained from a plugin’s metadata can contain null values and you should simply discard these null values without considering them for re-ordering plugins.