Just been to Anders Hejlsberg’s talk on the future of C#, where he outlined what’s coming in C# 4.0 and (some) of what might come in C# 5.0.

Importantly: C# 4.0 focuses on Dynamic Languages (i.e. the Dynamic Language Runtime (DLR))  and concurrent programming (i.e. programming for multi-core CPUs).

New in C# 4.0 is support for the attic type dynamic.
This allows you to specify a type which isn’t known until runtime.

Under the hood, it all seems to use the whole Type Invoke mechanism (i.e. reflection, which can be very slooooooow).
Which leads me to wonder: dynamics in C# 4.0 look like they’re cool in certain situations, but you end up with perf-problems, and the possibility for difficult-to-find runtime bugs.

For example, if I typed:

dynamic calc = GetCalculator();
int val = calc.App(2, 10);

Instead of

int val = calc.Add(2, 10);

well.. I won’t know that there is a  bug until I get to that line as it’s dynamically executed
At least, that’s my understanding.
It’ll be interesting to see how they address this.

In C# 5.0, Anders showed how they’re re-writing the C# compiler (csc) in managed code – and allowing you to interact with it from code.
Specifically, he showed how to dynamically generate, compile and execute code.. similar to what CodeDOM does today, but much much cooler.

Well, I’m at the Microsoft PDC (Professional Developers Conference) 2008 at the LA Convention Center in LA.

The keynote session is happening at the moment – Ray Ozzie has just announced Windows Azure, the new name for the cloud-services platform (I had heard that they would call is Windows Strata, but there you go!).

Azure look fairly cool, although the development environment is just VS.NET.

What’s new is that they host your services for you, including (in the future) SQL Services i.e. hosting your databases, which will be fairly cool.
When you create an Azure service in VS.NET, and then choose deploy, the service is packaged and you’re taking to the Azure Developers Portal, where the service is uploaded.

Bear in mind that this Microsoft’s answer to Amazon Web Services (AWS) and Google Apps – Microsoft are joining the party late, but I’m hoping that they have learnt something by being able to watch from the sidelines.

I’m also interested in all the Oslo sessions – for those of us in the BizTalk world, Oslo gives us information about what future versions of BizTalk will look like (post BTS 2009).
What’s important to realise is that Oslo is the code name for a whole suite of tools, as opposed to a single product.

I imagine this means that there will be a whole lot of products that come out under the Oslo codename/brand.

Aggregating messages is a fairly common task in BizTalk.
By "aggregating" I mean taking two separate messages with repeating elements and combining them into a new message which contains the elements of both messages - the same as doing a Union in SQL.

However, what if you want to remove duplicates?
It's not as easy as it seems, and in truth the only way I have found to do this is via custom XSLT.

Combining two messages
This is actually fairly easy: you use...

In this post:
   Methods in the helper class
   Notes on implementation
   Installation Instructions
   Problems with the Script Functoid
   Examples of Usage
   Download the code
   Testing the sample map

I’m not sure if this is actually of any use to anyone but if you do want to use Microsoft's Extended XPath Functions from a BizTalk Map (see my previous post) then I've created a wrapper assembly to do this for you.
Method names are the same as the original functions, but using "_" instead of "-".

It's not that well documented, but Microsoft have a bunch of extra functions you can use from XPath expressions within XSLT.
You can find the documentation on them here.

What isn't made apparent is that none of them work with XslTransform: a subset of them will work with XslCompiledTransform, and the others only work with MSXML 4.0 onwards.

This becomes obvious if you try and use these functions, or if you read this blog post from the Microsoft XML team on the XslCompiledTransform class.

Being the curious cat I am, I wondered if there was any way to use the functions from within XslTransform - if there was, I figured you could use the functions in a BizTalk Map...

If you prefer to do your reading offline, then you can download the entire series of 13 posts as a single 68-page Microsoft Word document, or Adobe PDF document.

Get them here:
     Understanding the BizTalk Mapper.doc (1.2MB)
     Understanding the BizTalk Mapper.pdf (640kB)

Please let me know what you think!

In this section:
Transformation Choices
BizTalk Mapper
Custom XSLT with the BizTalk Mapper
External Transform Engine
Transformation in code
Which one should you use?

Transformation Choices
When performing transformations in BizTalk, you have four choices (that I can think of):

1. Using the BizTalk Mapper
2. Using a custom XSLT file with the BizTalk Mapper
3. Using a separate transformation engine (called from code)
4. Performing transformations in code

Each of these offers their own benefits depending on your requirements.

Normally your choice will depend on 3 factors:

• Performance
• Complexity
• Maintainability

Generally you will get one (or two) of these, at the cost of the third.

For simple transformations, you can get all three with the Mapper using the built-in functoids.

Read more ...

In this section:
Performance
   Summary of Tests
   Testing performance in isolation (non-BizTalk)
   Performance Test Results
   Measuring Memory Usage in BizTalk
   BizTalk Memory Test Results
   Byte Arrays
   Analysing the performance results
Maintainability
   External XSLT
   Serialisable Classes
   Why is it so difficult to edit code in the Script functoid?
   Documentation

Any large BizTalk project will likely have had the inevitable conversations about performance and maintainability: will it be fast/sustainable enough, and will the tech support team (or whoever looks after the code once the developers have finished) be able to maintain it?

In this post I want to look at the performance of the Mapper, and also look at how maintainable maps are generally.

In order to do this, I want look at the different options you have for executing XSLT with the Mapper, and compare this to the most common non-Mapper mechanism for performing transformation: using serialisable classes.

Interestingly, all of the advanced functoids emit XSLT. No C# in sight at all.
The reason for this is that the functoids in this category all perform operations best suited to trees of data i.e. XML.
The only way to do this in C# would be to load the data into a DOM (i.e. XmlDocument) or XmlReader, or treat the XML as string data and search for tokens.

Note: this category was the one that actually started this series – I felt that if you knew the XSLT emitted by these functoids it would help understand when to use them, and what you can achieve with them.

Functoids covered in this category:

Assert	Record Count
Index	Scripting
Iteration	Table Looping
Looping	Table Extractor
Mass Copy	Value Mapping
Nil Value	Value Mapping (Flattening)

This category contains both Database and Cross Referencing Functoids – but they all connect to a database to retrieve/update data.
Unlike all other default functoids, these functoids all call classes/methods in external assemblies – no inline C# is emitted at all. Because of this, this is the only category that emits an ExtensionObjects file listing the strong names of the external assemblies used.

Note: in this category I show some of the source code from the external assemblies as well.

Functoids covered in this category:

Database Lookup	Get Common Value
Error Return	Remove Application ID
Format Message	Set Common ID
Get Application ID	Value Extractor
Get Application Value	Common Code
Get Common ID

Of the functoids in this category, only Cumulative Sum has a counterpart in XSLT v1.0 – all the others can be performed in XSLT v2.0, but not XSLT v1.0.

Functoids covered in this category:

Cumulative Average	Cumulative Minimum
Cumulative Concatenate	Cumulative Sum
Cumulative Maximum	Common Code

Yet another category which has no direct support in XSLT v1.0 or XSLT v2.0!
However, given the strong support for scientific functions in .NET, it's easy to call out to .NET classes, which is exactly what every single one of the functoids in this category does.

Having said that: have you ever used one of these functoids in a map? Care to share a real world example?
I'd be interested to find out how often they are used.

Functoids covered in this category:

10^n	Natural Logarithm
Arc Tangent	Sine
Base-Specified Logarithm	Tangent
Common Logarithm	X^Y
Cosine	Common Code
Natural Exponential Function

Surprisingly, neither XSLT v1.0 nor XSLT v2.0 have any built-in conversion support (well, not for the scenarios represented in this category anyway).
It is possible to download XSLT libraries which can do this sort of conversion (as mentioned in the notes below each functoid), but the XSLT is not pretty, and I'm not convinced about performance.
So C# is generally your only option here.

Functoids covered in this category:

ASCII to Character	Octal
Character to ASCII	Common Code
Hexadecimal

XSLT v1.0 has no support for Date/Time values, whilst XSLT v2.0 has full support.
Therefore it's not surprising that your only option is to use C#'s rich support for Date/Time values.
And this is why all of the functoids in this category emit inline C#.

Functoids covered in this category:

Add Days	Time
Date	Common Code
Date and Time

Whenever I've looked at the XSLT generated by a map I've always been confused by the amount of inline C# generated by these functoids.
After the String Functoids I'd say that these are the next most widely used and yet all but one of them has an XSLT v1.0 equivalent!
The code emitted for "Logical Equal" always makes me laugh – 12 lines of C# code can be replaced by... (wait for it)... one "=" symbol!

Functoids covered in this category:

Equal	Logical Existence
Greater Than	Logical NOT
Greater Than or Equal To	Logical Numeric
IsNil	Logical OR
Less Than	Logical String
Less Than or Equal To	Not Equal
Logical Date	Common Code

Mathematics is not a strong point of XSLT. XSLT v1.0 has very poor mathematic support, whilst XSLT v2.0 has better support, but only by a small amount. Therefore most of the functoids in this category can only be implemented in C#.

So if you want to perform a complicated mathematical function (i.e. anything more than addition or subtraction!) you’re better off using one of these functoids, or an external assembly.

Once again, inline C# isn’t the fastest, but given a choice between a slow function and no function, you might not have a choice.

Functoids covered in this category:

Absolute Value	Modulo
Addition	Multiplication
Division	Round
Integer	Square Root
Maximum Value	Subtraction
Minimum Value	Common Code

The String Functoids are probably the most frequently used in maps (in my experience), mainly because they're the most familiar to a procedural programmer (i.e. a C# or VB programmer). However because they all emit inline C#, they perform the slowest so if you want your maps to run faster you're better off using the corresponding XSLT, or implementing the functionality you require in a separate assembly.

Functoids covered in this category:

Lowercase	String Left Trim
Size	String Right
String Concatenate	String Right Trim
String Extract	Uppercase
String Find	Common Code
String Left

This whole series of posts started because I wanted to show what XSLT was emitted when using the default functoids provided by Microsoft.
Specifically, I wanted to show the XSLT emitted by the Advanced Functoids. Understanding this XSLT can help in understanding how to use the functoids.
For some reason (as seems to happen with me) the post expanded into a whole series on the Mapper... every time I explain one thing, I seem to want to explain all the things that the first thing is based on... oops.

Anyway, suffice to say that the next 9 posts will cover the code emitted by all of the default functoids provided with BizTalk 2004 / 2006 / 2006 R2.

One thing to realise is that the majority of the default functoids emit inline C# code – which is odd as quite a lot of the functionality can be performed using pure XSLT.

So for each functoid I've shown:

1. Whether XSLT or C# is emitted
2. Whether an XSLT equivalent exists
3. The XSLT or C# emitted by the functoid
4. Where C# is emitted, the equivalent XSLT to achieve the same functionality (in both XSLT v1.0 and v2.0)

In this post:
Introduction
BizTalk Mapper 101
History
BizTalk Mapper in BTS 2004 / 2006 / 2006R2
What happens when a map is compiled
What happens when a map is executed
XslTransform vs XslCompiledTransform
XSLT 1.0 vs XSLT 2.0

Introduction
The BizTalk Mapper is an integral part of the BTS toolkit and, depending on your inclination and experience, you will probably either love, hate it, or not care about it.
At its core, the BizTalk Mapper is simply a visual tool for specifying XSLT (eXtensible Stylesheet Language Transformations).
This XSLT is used to transform one stream of XML data into another stream of XML data. And since a "stream of XML data" = "message in BizTalk", this means transforming one message into another message.

[Aside: you might think that the Mapper allows you to specify multiple source or destination messages, but this is actually a trick - BizTalk creates a special schema with one part per input/output message - there's still only one input/output message]

How well the BizTalk Mapper performs its task depends on the complexity of the transformation you're attempting - and the skill/experience of the developer using the BizTalk Mapper!

For those of you who are unaware, XPath is a powerful query language for XML (ignoring XQuery for now).

In my experience I've worked with a lot of BizTalk developers who have come from a C# (and usually VB before that) background, and so haven't always had the background in XML, XSLT, XSD, and XPath needed to fully appreciate BizTalk.

Because let's face it: before you were working on BizTalk, how often did you need to deal directly with Xml? When writing ASP.NET web services, the .NET framework abstracts away the need to understand XML, leaving you with classes which are handily serialised/de-serialised across the wire for you.

Note: I have come across a similar pattern in the J2EE world – ask many Java developers about an Xml document they received as a request in a J2EE Web Service, and they'll only know about Beans, as the framework looks after the de-serialization from Xml into a Bean for them).

However, BizTalk is a different beastie: it works predominantly with data streams containing Xml.
Therefore the ability to know how to query this Xml without having to resort to C# code is very very important.

BizTalk 2004 and 2006/2006R2 support XPath 1.0 - as at this time, there is no support for XPath 2.0 in BizTalk.
You can get a good idea of the functions available in XPath 1.0 here (which is a useful summary of the W3 Recommendation)

(as a comparison, here's what XPath 2.0 gives you - here's hoping we get this in BizTalk sometime soon...!)

This post follows on from yesterday's post: Creating BizUnit Test Cases for comparing Xml Files

If you are using the XmlValidationStep/XmlValidationStepEx BizUnit steps and using XPath validation, then it can be a pain to write all these XPathValidation statements by hand.

So I wrote a utility to generate the XPath statements for you.
In fact, it generates the entire BizUnit Test Case for validating an Xml file.
Once it has been generated, all you have to do is[...]