Understanding the BizTalk Mapper: Part 13 - Is the Mapper the best choice for Transformation in BizTalk?#
In this section:
Transformation Choices
BizTalk Mapper
Custom XSLT with the BizTalk Mapper
External Transform Engine
Transformation in code
Which one should you use?

Transformation Choices
When performing transformations in BizTalk, you have four choices (that I can think of):
  1. Using the BizTalk Mapper
  2. Using a custom XSLT file with the BizTalk Mapper
  3. Using a separate transformation engine (called from code)
  4. Performing transformations in code
Each of these offers their own benefits depending on your requirements.

Normally your choice will depend on 3 factors:
  • Performance
  • Complexity
  • Maintainability
Generally you will get one (or two) of these, at the cost of the third.

For simple transformations, you can get all three with the Mapper using the built-in functoids.

Friday, February 29, 2008 4:05:35 PM (GMT Standard Time, UTC+00:00) #    Comments [3]  |  Trackback

 

Understanding the BizTalk Mapper: Part 12 - Performance and Maintainability#
In this section:
Performance
   Summary of Tests
   Testing performance in isolation (non-BizTalk)
   Performance Test Results
   Measuring Memory Usage in BizTalk
   BizTalk Memory Test Results
   Byte Arrays
   Analysing the performance results
Maintainability
   External XSLT
   Serialisable Classes
   Why is it so difficult to edit code in the Script functoid?
   Documentation

Any large BizTalk project will likely have had the inevitable conversations about performance and maintainability: will it be fast/sustainable enough, and will the tech support team (or whoever looks after the code once the developers have finished) be able to maintain it?

In this post I want to look at the performance of the Mapper, and also look at how maintainable maps are generally.

In order to do this, I want look at the different options you have for executing XSLT with the Mapper, and compare this to the most common non-Mapper mechanism for performing transformation: using serialisable classes.
Tuesday, February 26, 2008 7:14:07 PM (GMT Standard Time, UTC+00:00) #    Comments [1]  |  Trackback

 

Understanding the BizTalk Mapper: Part 11 - Advanced Functoids#
Interestingly, all of the advanced functoids emit XSLT. No C# in sight at all.
The reason for this is that the functoids in this category all perform operations best suited to trees of data i.e. XML.
The only way to do this in C# would be to load the data into a DOM (i.e. XmlDocument) or XmlReader, or treat the XML as string data and search for tokens.

Note: this category was the one that actually started this series – I felt that if you knew the XSLT emitted by these functoids it would help understand when to use them, and what you can achieve with them.

Functoids covered in this category:
Assert Record Count
Index Scripting
Iteration Table Looping
Looping Table Extractor
Mass Copy Value Mapping
Nil Value Value Mapping (Flattening)
Wednesday, February 20, 2008 3:21:53 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 10 - Database Functoids#
This category contains both Database and Cross Referencing Functoids – but they all connect to a database to retrieve/update data.
Unlike all other default functoids, these functoids all call classes/methods in external assemblies – no inline C# is emitted at all. Because of this, this is the only category that emits an ExtensionObjects file listing the strong names of the external assemblies used.

Note: in this category I show some of the source code from the external assemblies as well.

Functoids covered in this category:
Database Lookup Get Common Value
Error Return Remove Application ID
Format Message Set Common ID
Get Application ID Value Extractor
Get Application Value Common Code
Get Common ID  

Monday, February 18, 2008 3:20:59 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 9 - Cumulative Functoids#
Of the functoids in this category, only Cumulative Sum has a counterpart in XSLT v1.0 – all the others can be performed in XSLT v2.0, but not XSLT v1.0.

Functoids covered in this category:
Cumulative Average Cumulative Minimum
Cumulative Concatenate Cumulative Sum
Cumulative Maximum Common Code

Saturday, February 16, 2008 3:20:01 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 8 - Scientific Functoids#
Yet another category which has no direct support in XSLT v1.0 or XSLT v2.0!
However, given the strong support for scientific functions in .NET, it's easy to call out to .NET classes, which is exactly what every single one of the functoids in this category does.

Having said that: have you ever used one of these functoids in a map? Care to share a real world example?
I'd be interested to find out how often they are used.

Functoids covered in this category:
10^n Natural Logarithm
Arc Tangent Sine
Base-Specified Logarithm Tangent
Common Logarithm X^Y
Cosine Common Code
Natural Exponential Function  

Friday, February 15, 2008 3:19:04 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 7 - Conversion Functoids#
Surprisingly, neither XSLT v1.0 nor XSLT v2.0 have any built-in conversion support (well, not for the scenarios represented in this category anyway).
It is possible to download XSLT libraries which can do this sort of conversion (as mentioned in the notes below each functoid), but the XSLT is not pretty, and I'm not convinced about performance.
So C# is generally your only option here.

Functoids covered in this category:
ASCII to Character Octal
Character to ASCII Common Code
Hexadecimal  

Thursday, February 14, 2008 3:18:07 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 6 - Date/Time Functoids#
XSLT v1.0 has no support for Date/Time values, whilst XSLT v2.0 has full support.
Therefore it's not surprising that your only option is to use C#'s rich support for Date/Time values.
And this is why all of the functoids in this category emit inline C#.

Functoids covered in this category:
Add Days Time
Date Common Code
Date and Time  

Wednesday, February 13, 2008 3:17:24 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 5 - Logical Functoids#
Whenever I've looked at the XSLT generated by a map I've always been confused by the amount of inline C# generated by these functoids.
After the String Functoids I'd say that these are the next most widely used and yet all but one of them has an XSLT v1.0 equivalent!
The code emitted for "Logical Equal" always makes me laugh – 12 lines of C# code can be replaced by... (wait for it)... one "=" symbol!

Functoids covered in this category:
Equal Logical Existence
Greater Than Logical NOT
Greater Than or Equal To Logical Numeric
IsNil Logical OR
Less Than Logical String
Less Than or Equal To Not Equal
Logical Date Common Code

Tuesday, February 12, 2008 3:16:22 PM (GMT Standard Time, UTC+00:00) #    Comments [1]  |  Trackback

 

Understanding the BizTalk Mapper: Part 4 - Mathematical Functoids#
Mathematics is not a strong point of XSLT. XSLT v1.0 has very poor mathematic support, whilst XSLT v2.0 has better support, but only by a small amount. Therefore most of the functoids in this category can only be implemented in C#.

So if you want to perform a complicated mathematical function (i.e. anything more than addition or subtraction!) you’re better off using one of these functoids, or an external assembly.

Once again, inline C# isn’t the fastest, but given a choice between a slow function and no function, you might not have a choice.

Functoids covered in this category:
Absolute Value Modulo
Addition Multiplication
Division Round
Integer Square Root
Maximum Value Subtraction
Minimum Value Common Code

Monday, February 11, 2008 3:13:46 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 3 - String Functoids#
The String Functoids are probably the most frequently used in maps (in my experience), mainly because they're the most familiar to a procedural programmer (i.e. a C# or VB programmer). However because they all emit inline C#, they perform the slowest so if you want your maps to run faster you're better off using the corresponding XSLT, or implementing the functionality you require in a separate assembly.

Functoids covered in this category:
Lowercase String Left Trim
Size String Right
String Concatenate String Right Trim
String Extract Uppercase
String Find Common Code
String Left  
Saturday, February 09, 2008 12:28:03 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 2 - Functoids Overview#
This whole series of posts started because I wanted to show what XSLT was emitted when using the default functoids provided by Microsoft.
Specifically, I wanted to show the XSLT emitted by the Advanced Functoids. Understanding this XSLT can help in understanding how to use the functoids.
For some reason (as seems to happen with me) the post expanded into a whole series on the Mapper... every time I explain one thing, I seem to want to explain all the things that the first thing is based on... oops.

Anyway, suffice to say that the next 9 posts will cover the code emitted by all of the default functoids provided with BizTalk 2004 / 2006 / 2006 R2.

One thing to realise is that the majority of the default functoids emit inline C# code – which is odd as quite a lot of the functionality can be performed using pure XSLT.

So for each functoid I've shown:
  1. Whether XSLT or C# is emitted
  2. Whether an XSLT equivalent exists
  3. The XSLT or C# emitted by the functoid
  4. Where C# is emitted, the equivalent XSLT to achieve the same functionality (in both XSLT v1.0 and v2.0)
Friday, February 08, 2008 12:03:15 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Understanding the BizTalk Mapper: Part 1 - Introduction#
In this post:
Introduction
BizTalk Mapper 101
History
BizTalk Mapper in BTS 2004 / 2006 / 2006R2
What happens when a map is compiled
What happens when a map is executed
XslTransform vs XslCompiledTransform
XSLT 1.0 vs XSLT 2.0

Introduction
The BizTalk Mapper is an integral part of the BTS toolkit and, depending on your inclination and experience, you will probably either love, hate it, or not care about it.
At its core, the BizTalk Mapper is simply a visual tool for specifying XSLT (eXtensible Stylesheet Language Transformations).
This XSLT is used to transform one stream of XML data into another stream of XML data. And since a "stream of XML data" = "message in BizTalk", this means transforming one message into another message.

[Aside: you might think that the Mapper allows you to specify multiple source or destination messages, but this is actually a trick - BizTalk creates a special schema with one part per input/output message - there's still only one input/output message]

How well the BizTalk Mapper performs its task depends on the complexity of the transformation you're attempting - and the skill/experience of the developer using the BizTalk Mapper!

Thursday, February 07, 2008 11:38:44 AM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

XPath – the hidden language of BizTalk?#

For those of you who are unaware, XPath is a powerful query language for XML (ignoring XQuery for now).

In my experience I've worked with a lot of BizTalk developers who have come from a C# (and usually VB before that) background, and so haven't always had the background in XML, XSLT, XSD, and XPath needed to fully appreciate BizTalk.

Because let's face it: before you were working on BizTalk, how often did you need to deal directly with Xml? When writing ASP.NET web services, the .NET framework abstracts away the need to understand XML, leaving you with classes which are handily serialised/de-serialised across the wire for you.

Note: I have come across a similar pattern in the J2EE world – ask many Java developers about an Xml document they received as a request in a J2EE Web Service, and they'll only know about Beans, as the framework looks after the de-serialization from Xml into a Bean for them).

However, BizTalk is a different beastie: it works predominantly with data streams containing Xml.
Therefore the ability to know how to query this Xml without having to resort to C# code is very very important.

BizTalk 2004 and 2006/2006R2 support XPath 1.0 - as at this time, there is no support for XPath 2.0 in BizTalk.
You can get a good idea of the functions available in XPath 1.0 here (which is a useful summary of the W3 Recommendation)

(as a comparison, here's what XPath 2.0 gives you - here's hoping we get this in BizTalk sometime soon...!)

Friday, November 30, 2007 4:53:03 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Auto-generating BizUnit Test Cases#

This post follows on from yesterday's post: Creating BizUnit Test Cases for comparing Xml Files

If you are using the XmlValidationStep/XmlValidationStepEx BizUnit steps and using XPath validation, then it can be a pain to write all these XPathValidation statements by hand.

So I wrote a utility to generate the XPath statements for you.
In fact, it generates the entire BizUnit Test Case for validating an Xml file.
Once it has been generated, all you have to do is[...]

Tuesday, November 27, 2007 5:59:18 PM (GMT Standard Time, UTC+00:00) #    Comments [1]  |  Trackback

 

Creating BizUnit Test Cases for comparing Xml Files#

BizUnit is a great tool for performing end-to-end testing of BizTalk applications (and can even be used for testing non-BizTalk applications like Web services too).
Although you're not strictly performing unit-testing of your BizTalk artefacts per-se, with the right setup you can get very close to unit testing.

For example if you want to test an orchestration in isolation you can:

  1. Hook your orchestration up to file send/receive ports
    or
  2. Write a façade orchestration which calls your orchestration, and is itself hooked up to file send/receive ports or
    or
  3. Write your own orchestration hosting engine

I don't know many people who'd be crazy enough to do 3) (although I do know one…) but BizUnit is of great help for 1) and 2)

In its simplest form you can use BizUnit to[...]

Monday, November 26, 2007 5:51:27 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

DanSharp XmlViewer#

Updates:
19/04/2018
The source code and MSIs for this tool (inc v1.1) have moved to GitHub here: https://github.com/probertdaniel/dansharpxmlviewer

03/10/2011
The source code and MSIs for this tool (inc v1.1) are now available on CodePlex here: http://dansharpxmlviewer.codeplex.com

22/01/2008
I'll release v1.1 this week - only has minor bug fixes (one of them related to problems with opening files from the recently used file menu).
Also I'm planning to release the source code this weekend.


So here's a little utility that I've been using for myself for a while, but thought I'd release in case it's of use to anyone else.

There are three tasks I perform quite regularly when doing XML or BizTalk development:

  1. Building and testing the XPath needed to retrieve a value from a given request/response document
  2. Validating a document against a schema
  3. Generating BizUnit tests cases (from an Xml instance doc, creating XPath statements to validate for all the data in the document)

This tool does all three of these things:


XmlViewer main form


Generating a BizUnit Test Case

Although there are good tools on the market for doing 1. and 2. (e.g. Altova XmlSpy and Stylus Studio) most of them require installation and are not free.
And none of them generate BizUnit Test Cases!
I wanted a single EXE which would do the most common tasks for me.

Also note that Visual Studio will give you the XPath for an element/attribute in an Xsd, but not an Xml document.
And although you can validate a document in Visual Studio, it's a hassle each time you want to change the Xml document used (right click schema, choose properties, change Input Instance Filename, cick OK, right click schema, choose Validate Instance).

Having a scratch pad to play with XPath queries has been very useful, and is the original reason I wrote this.
But generating BizUnit Test Cases has so far been the most useful... if you use the REG file provided, you can even right-click an Xml file to generate a BizUnit TestCase for it.

There is a command line version supplied as well, so if you need to generate a lot of BizUnit Test Cases, you can do so in a batch file/script.

The utility is written as a single EXE, with no additional files needed (other than .NET 2.0) and it doesn't need to be installed.
I have a copy on my USB key and I run it from there.

Caveat: I use this tool every day, so most bugs have (hopefully) been eradicated.
So my apologies if you come across something I haven't tested yet - please let me know if you do.

Files available to download:
Note: I will be making the source code for this utility available, but need to tidy it up a bit first.
Will add it here when it's ready - probably in a week or so.

MSI Installer (contains all of the below files)
    DanSharp.XmlViewer.Setup.msi (1.8MB)

Executable (Windows version)
    XmlViewer.exe (80kb)

Executable (Command line version)
    XmlViewer.com (80kb)

Registry File (.reg) for adding right-click support to Xml files
    GenerateBizUnitTestCase.reg (2kb)

Documentation
    DanSharp XmlViewer Documentation v1.0.pdf (860kb PDF)
    DanSharp XmlViewer Documentation v1.0.doc (600kb Word 97 Doc)

Source Code
   (Coming soon)

If you come across bugs or have any comments, then leave a comment here, or email me at:
xmlviewerATprobertsolutions.com

Sunday, November 25, 2007 5:45:14 PM (GMT Standard Time, UTC+00:00) #    Comments [1]  |  Trackback

 

BizUnit: XmlValidationStep using .NET 2.0 Schema Validation#
This is something that bit me recently:
The current version of BizUnit (v.2.2.1003.0)  uses the .NET 1.0 XmlValidatingReader to do schema validation in both the XmlValidationStep and XmlValidationStepEx steps.

.NET 2.0 deprecated the XmlValidatingReader class, proposing that you use the XmlReader class and the new XmlReaderSettings class instead.
Microsoft also introduced a slew of bug fixes to the validation logic[...]
Saturday, November 24, 2007 6:43:10 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Distinguished Fields and Optional Elements#

Thought this might be of interest to a few BizTalk developers out there.

It's quite common practice to use Distinguished Fields in orchestrations, to get/set the value of an element or attribute.

However, if the element you're trying to set/get doesn't exist, then this poses a few problems.

Setting a Distinguished Field
If the element/attribute doesn't exist in the target message, then you'll get an exception, no questions asked – there's no way around this.
Under the covers, the SetDistinguishedField method is called to set the value – and it doesn't check if the element/attribute exists first.
The same thing happens if you use the xpath() function to set a value, and the element/attribute doesn't exist – BizTalk isn't about to modify your message and add the element/attribute for you[...]

Thursday, November 15, 2007 3:09:42 PM (GMT Standard Time, UTC+00:00) #    Comments [0]  |  Trackback

 

Manually sending a response back to a waiting Web Service client#

(or how to do asynchronous processing of a synchronous request/response message)
[Note: this post follows on from my previous post How to validate Xml Documents against schemas in BizTalk. For example, if you validate a message in a pipeline and validation fails, how do you send a response back to a waiting web service client.]

For a while now, people have been struggling with how to asynchronously process request/response messages - basically, how to get away from having a request/response port in an orchestration bound to a request/response  receive location.

For example, Yossi Dohan blogged on this a while back, and came up with a solution involving multiple orchestrations:
http://www.sabratech.co.uk/blogs/yossidahan/2006/06/sync-to-async-conversion.html

The problem revolves around the fact that BizTalk will always demote the EpmRRCorrelationToken context property when your message leaves the orchestration if you try and manually set it yourself.

Sunday, September 09, 2007 11:41:48 AM (GMT Daylight Time, UTC+01:00) #    Comments [1]  |  Trackback

 

All content © 2020, Daniel Probert
On this page
This site
Calendar
<February 2008>
SunMonTueWedThuFriSat
272829303112
3456789
10111213141516
17181920212223
2425262728291
2345678
Archives
Sitemap
Blogroll OPML
Disclaimer

Powered by: newtelligence dasBlog 2.3.12105.0

The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

Send mail to the author(s) E-mail

Theme design by Jelle Druyts


Pick a theme: